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ABSTRACT 


Prediction  markets  generally  are  small-scale  electronic  markets  that  tie  payoffs  to 
measurable  future  events.  They  are  similar  to  stock  markets,  where  the  “stocks”  are 
outcomes  or  events  rather  than  shares  in  a  company.  The  growing  popularity  of 
prediction  markets  reflects  the  notion  that  markets  are  an  excellent  means  of  efficient 
information  aggregation  among  a  disparate  group  of  people.  Trading  prices  in  the 
prediction  markets  provide  decision  makers  with  a  timely,  accurate,  and  continuously 
updated  picture  on  the  likelihood  of  future  events.  This  enables  decision  makers  to  better 
evaluate  risk.  Based  on  historical  successes  in  prediction  market  utilization,  it  is  both 
logical  and  important  to  assess  the  usefulness  of  prediction  markets  in  contributing  to 
critical  elements  of  Navy  total  force  shaping. 

Navy  Manpower,  Personnel,  Training,  and  Education  (Nl)  regularly  forecasts  re¬ 
enlistment  rates,  over/under  endstrength,  and  many  other  force-shaping  factors  as  an 
input  into  their  resource  allocation  decision-making  process.  In  an  effort  to  improve  upon 
the  force-shaping  decision-making  process,  Nl  has  shown  interest  in  using  prediction 
markets  to  complement  or  replace  alternative  methods  for  forecasting  various  Navy 
force- shaping  elements. 

The  aim  of  this  thesis  is  to  act  as  a  foundation  for  ongoing  prediction  market 
research  within  the  Department  of  Defense  (DoD). 
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I.  INTRODUCTION 


A.  BACKGROUND  AND  PURPOSE 

Prediction  markets  generally  are  small-scale  electronic  markets  that  tie  payoffs  to 
measurable  future  events.  They  are  similar  to  stock  markets,  where  the  “stocks”  are 
outcomes  or  events  rather  than  shares  in  a  company.  Private  firms  are  using  prediction 
markets  to  help  forecast  key  events,  such  as  the  success  of  new  products,  future 
profitability,  or  mergers/acquisitions  within  their  industry.  Prediction  markets  also  have 
been  popular  as  an  alternative  to  polls  or  surveys.  The  growing  popularity  of  prediction 
markets  reflects  the  notion  that  markets  are  an  excellent  means  of  efficient  and  effective 
information  aggregation  among  a  disparate  and  diverse  group  of  people.  The  strength  of 
prediction  markets  is  that  they  aggregate  knowledge  in  a  clever  way,  allowing  people  to 
digest  other  people’s  information  and  make  their  own  decision  based  upon  the  new 
information,  vice  simply  averaging  survey  responses.  The  trading  prices  in  the  prediction 
markets  provide  decision  makers  with  a  timely,  accurate,  and  continuously  updated 
picture  on  the  likelihood  of  future  events.  This  enables  decision  makers  to  evaluate  risk, 
and  it  provides  an  early  warning  of  possible  problems  and  needed  policy  changes.  Based 
on  private  and  commercial  successes  in  prediction  market  utilization,  it  is  both  logical 
and  important  to  assess  the  usefulness  of  prediction  markets  in  contributing  to  critical 
elements  of  Navy  total  force  shaping. 

Navy  Manpower,  Personnel,  Training,  and  Education  (Nl)  regularly  forecasts 
future  attrition  rates,  re-enlistment  rates,  over/under  strength,  and  many  other  force¬ 
shaping  factors  as  an  input  into  their  resource  allocation  decision-making  process.  Nl 
relies  on  historically  based  econometrics  in  tandem  with  quick-poll  data  to  forecast  force¬ 
shaping  estimates.  Econometrics  models,  however,  often  produce  incorrect  predictions. 

To  improve  upon  the  force-shaping  decision-making  process,  Nl  has  shown 
interest  in  using  prediction  markets  to  replace  or  complement  quick-polls  and 
econometrics  for  various  aspects  of  force-shaping  forecasting.  The  significant  potential  of 
prediction  markets  lies  in  their  ability  to  efficiently  and  accurately  aggregate  current 
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information  in  any  economic  environment.  Furthermore,  prediction  markets  may  offer 
potential  cost-savings  against  methods  such  as  polling. 

The  objectives  of  this  research  paper  are:  1)  to  consolidate  previous  prediction 
market  literature  to  provide  a  thorough  description  and  assessment  of  prediction  market 
benefits,  limitations,  and  design  and  implementation;  and  2)  to  consider  the  feasibility  of 
utilizing  prediction  markets  as  alternative  forecasting  measures  for  Navy  organizational 
leadership  decision  making.  As  part  of  these  objectives,  we  address  the  following  issues 
throughout: 

B.  RESEARCH  QUESTIONS 

1.  What  are  the  key  benefits  and  potential  limitations  of  prediction  markets? 

2.  What  does  the  current  literature  and  practical  evidence  suggest  about 
prediction  market  use  as  a  forecasting  tool? 

3.  Can  prediction  markets  be  an  effective  tool  for  Navy  force-shaping 
forecasting? 

4.  In  what  areas  should  the  Navy  consider  using  a  prediction  market? 

5.  In  conducting  a  pilot  prediction  market,  what  lessons  learned  can  we 
provide  to  the  Navy  regarding  the  design,  implementation,  and  utilization 
of  prediction  markets? 

C.  METHODOLOGY 

This  research  provides  a  thorough  review  and  overview  of  previous  prediction 
market  literature.  Furthermore,  it  underscores  the  key  benefits  and  limitations  of 
prediction  markets,  discusses  when  prediction  markets  are  best  utilized,  and  addresses 
their  critical  design  and  implementation  issues.  To  better  understand  the  intricacies 
associated  with  market  design  and  implementation,  a  practical  experimental  prediction 
market  was  conducted.  Finally,  using  the  research  and  experimental  lessons  learned  as  a 
basis,  this  thesis  offers  recommendations  on  the  potential  forecasting  usefulness  of 
prediction  markets  for  critical  elements  of  Navy  total  force-shaping  strategy  and  policies. 
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D.  LIMITATIONS 


This  research  project  is  limited  in  scope  and  breadth  due  to  time  and  participatory 
constraints  associated  with  designing  and  conducting  an  experimental  prediction  market. 
Furthermore,  the  experiment’s  participant  pool  perhaps  was  not  as  broad  and  diverse  as 
preferred  for  a  market  of  such  specificity.  Finally,  due  to  time  and  budget  constraints,  it 
was  not  feasible  to  ensure  all  participants  were  trained  properly  and  collectively  on  the 
concept  and  nuances  of  prediction  markets.  ^ 


^  Multiple  anonymous  post-market  survey  responses  indicate  that  several  initial  participants  ceased 
participation  due  to  a  lack  of  understanding.  This  likely  is  a  result  of  improper  participant  training. 
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II.  PREDICTION  MARKETS  SYNOPSIS 


This  introduction  provides  an  overview  of  prediction  markets  and  serves  as  a 
prediction  market  primer  for  future  practitioners.  Prediction  markets  (also  known  as 
information,  electronic  or  decision  markets)  are  proposed  as  an  alternative  forecasting 
means  for  predicting  future  events.  The  idea  is  for  prediction  markets  to  supplement 
existing  forecasting  techniques,  such  as  quantitative  and  judgmental  methods,  when  these 
techniques  are  unsuitable  for  predicting  future  events. 

Quantitative  and  judgmental  forecasting  methods  have  limitations.  Quantitative 
methods  rely  on  historical  data.  These  statistical  methods  are  invalid  when  organizations 
simply  have  not  collected  the  historical  data  needed  to  predict  future  events  or  the 
fundamental  nature  of  the  operating  environment  shifts  radically  and  invalidates  implicit 
assumptions.  These  situations  render  quantitative  forecasts  meaningless.  Similarly, 
judgmental  methods  also  have  practical  limitations.  The  Delphi  and  judgmental 
bootstrapping  methods  are  the  most  prominent  judgmental  forecasting  methods  and 
combine  the  opinions  of  experts.  However,  identifying  experts,  gamering  their 
participation,  and  making  sense  of  conflicting  subjective  opinions  can  be  a  herculean  task 
(Ashton  &  Ashton,  1985;  Batchelor  &  Dua,  1995).  These  practical  limitations  provide  an 
opportunity  for  an  alternative  forecasting  method. 

A.  DEFINITION 

Prediction  markets  do  not  have  a  universal  name  or  definition.  However,  an 
extended  literature  review  of  prediction  markets  by  Tziralis  &  Tatsiopoulos  (2007)  chose 
this  definition  based  upon  a  definition  by  Berg,  Nelson,  and  Rietz  (2003): 

Prediction  markets  are  defined  as  markets  that  are  designed  and  run  for  the 
primary  purpose  of  mining  and  aggregating  information  scattered  among 
traders  and  subsequently  using  this  information  in  the  form  of  market 
values  in  order  to  make  predictions  about  specific  future  events,  (p.  1) 
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B.  THEORY 


Friedrich  Hayek’s  article  “The  Use  of  Knowledge  in  Society”  (1945)  describes 
the  theoretical  superiority  of  aggregating  widely  dispersed  information  versus  relying 
solely  on  statistical  algorithms  or  experts.  Hayek  emphasizes  the  intrinsic  value  of 
implicit  information,  what  is  now  referred  to  as  tacit  knowledge.  Simply  put,  it  takes  time 
to  learn  how  to  perform  a  job  well  even  after  receiving  formal,  theoretical  education. 
Additionally,  he  argues  from  a  resource  allocation  perspective  that  a  centralized  planning 
office  cannot  meet  the  efficiency  of  an  open  market  because  the  individuals  only  know  a 
very  small  amount  compared  to  the  collective  knowledge  of  society.  Therefore,  a 
decentralized  economic  structure  complements  the  dispersed  nature  of  information  spread 
throughout  society  (p.  520).  Additionally,  he  challenged  himself  and  his  peers  to  find  a 
method  of  pulling  together  this  knowledge  from  the  disparate  sources. 

Prediction  markets  are  based  upon  the  notion  of  collective  intelligence.  James 
Surowiecki’s  book  The  Wisdom  of  Crowds  re-popularized  Hayek’s  ideas  in  2004.  The 
book  presents  a  case  that,  under  the  right  circumstances,  the  collective  knowledge  of  a 
group  can  be  superior  to  the  smartest  individual  members  of  the  group.  Surowiecki  states 
four  criteria  are  required  for  groups  to  demonstrate  collective  intelligence  (2004,  pp. 
XVIII-XIX): 

•  Diversity  of  opinion 

•  Member  independence 

•  Decentralization 

•  A  mechanism  to  aggregate  opinions 

Prediction  markets  are  proven  information  aggregation  mechanisms.  Moreover, 
previous  empirical  evidence  supports  the  position  that  prediction  markets  are  more 
accurate  than  quantitative  and  judgmental  methods  at  predicting  the  likelihood  of  future 
events.  Herein  lies  the  competitive  advantage  for  prediction  markets.  Subsequent  chapters 
discuss  prediction  market  accuracy  in  greater  depth. 
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c. 


PREDICTION  MARKET  ANATOMY 


This  section  will  provide  a  brief  framework  to  increase  readers’  understanding 
regarding  how  prediction  markets  work.  This  will  serve  as  a  brief  introduction  to  some  of 
the  design  factors  and  terminology;  however,  these  concepts  will  be  covered  in  detail  in 
subsequent  chapters. ^ 


I.  Design  Factors 

a.  Claim  Definition 

The  claim  definition  is  the  question  or  statement  posed  to  the  marketplace 
traders.  The  claim  definition  must  account  for  all  possible  outcomes. 


b.  Claim  Structure 


The  claim  structure  is  the  means  of  assigning  the  associated  payoff  to 
market  traders.  There  are  three  main  types  of  claim  structures. 

(1)  Winner-take-all 

•  Pays  out  $1  if  a  specific  event  occurs 

•  Pays  $0  otherwise 

•  This  claim  structure  is  useful  for  determining  the  probability  of  whether  an 
event  will  or  will  not  occur. 

(2)  Index 

•  Contract  pays  out  the  value  of  a  specific  future  event 

•  This  claim  structure  is  useful  for  determining  the  mean  value  of  the 
market’s  expectation  of  a  continuous  number  range. 

(3)  Spread 

•  Amount  of  bet  is  fixed 

•  Market  trades  based  on  cutoffs  that  determine  whether  event  occurs 

•  This  claim  structure  is  useful  for  determining  the  median  expected 
probability  that  an  event  will  or  will  not  occur. 
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A  focused  discussion  on  each  design  topic  below  is  provided  in  Chapter  V. 
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c.  Trading  Mechanisms 

The  market  trading  mechanism  is  what  determines  market  conduct  among 
all  participants;  it  is  that  which  connects  buyers  and  sellers  for  market  trading.  There  are 
three  common  trading  mechanisms: 

(1)  Continuous  Double  Auction  (CD A) 

•  Person-to-person  trading 

(2)  Market  Scoring  Rules  (MSR) 

•  Mathematical  algorithm  to  facilitate  liquidity  by  use  of  an  automated 
market  maker^ 

•  Maintains  a  probability  distribution  across  all  possible  events 

(3)  Dynamic  Pari-mutuel  (DPM) 

•  Mathematical  algorithm  to  facilitate  liquidity  by  use  of  an  automated 
market  maker 

•  Operates  based  on  a  price  function  that  reflects  a  continuously  updated 
probability  for  a  given  event’s  occurrence 

d.  Participation 

Active  and  productive  participation  is  an  essential  element  of  prediction 
markets.  Wider  participation  pools  generally  provide  a  greater  knowledge  base  than  do 
narrow  participation  pools.  Thin  markets  exist  when  there  are  few  participants  and/or 
participants  do  not  actively  participate  in  such  a  way  to  ensure  continuous  trading  occurs. 
Thin  markets  are  a  limitation  of  prediction  markets.  Prediction  market  designers  hope  for 
thick  markets. 


e.  Real  Money,  Play  Money  and  Associated  Incentives 

Incentives  can  be  useful  for  enticing  participants  who  otherwise  would  not 
engage  in  the  prediction  market.  Cash  incentives  are  most  commonly  associated  with  real 
money  markets.  Play  money  markets  are  among  the  most  prevalent  for  public  and 


3  Automated  market  makers  are  explained  in  subsequent  chapters;  refer  to  page  42  for  a  more  thorough 
discussion,  under  Trading  Mechanisms. 
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professional  use.  These  markets  usually  provide  some  initial  endowment  of  play  money. 
Generally,  play  money  markets  rely  on  the  use  of  other  incentives,  such  as  prizes  or 
points,  to  elicit  participation  (Chen,  2005). 

D.  HOW  PREDICTION  MARKETS  WORK 

In  finance,  the  efficient-market  hypothesis  posits  open  financial  markets 
efficiently  aggregate  all  available  information  into  market  prices.  These  market  prices 
reflect  all  known  information  and  quickly  respond  to  new  information  leading  to  a 
change  in  price.  Organizations  are  applying  this  concept  to  aggregate  organizational 
information  by  using  their  disparate  and  diverse  stakeholder  group  as  prediction  market 
traders.  Thus,  prediction  markets  offer  a  real-time  mechanism  to  aggregate  this 
fragmented  information  for  organizations  to  use  it  to  their  advantage.  The  commodities 
traded  are  claims  about  future  events  in  which  the  market  prices  represent  a  real-time, 
consensus  probability  about  the  event's  likelihood  of  occurrence.  The  market  also 
provides  a  historical  context  in  which  one  may  see  how  individual  and  collective 
opinions  have  changed  over  time. 

In  general,  the  trading  price  depends  on  the  joint  distribution  of  the  traders’ 
beliefs,  budgets  and  risk  preferences  (Manski,  2006).  Budgets  are  intuitively  less  of  a 
consideration  when  the  market  uses  play  money  as  currency.  Similarly,  play  money 
traders  may  be  more  apt  to  take  risks  that  they  otherwise  would  not  take  with  real  money. 
Fortunately,  empirical  evidence  suggests  the  accuracy  of  prediction  markets  using  play 
money  is  comparable  to  real  money  prediction  markets  (Servan-Schreiber,  Wolfers, 
Pennock,  &  Galebach,  2004).  This  same  study  concludes  that  the  use  of  real  money  is  not 
critical  but  is  one  of  many  ways  to  motivate  well-informed  people  to  trade  in  the  market 
(p.  250). 

The  evidence  is  mixed  on  the  issue  of  market  accuracy  when  using  real  versus 
play  money.  Rosenbloom  and  Notz  (2006)  conducted  another  study  in  response  to  the 
aforementioned  study’s  assertion  that  play  money  market  accuracy  is  comparable  to  real 
money  market  accuracy.  Although  they  found  that  both  real  and  play  money  markets 
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provide  reasonably  aeeurate  foreeasts,  their  major  elaim  is  “that  real  money  markets  are 
significantly  more  accurate  than  play  money  markets  when  forecasting  non-sports 
events”  (p.  63). 

Trading  prices  do  not  simply  reflect  an  average  assessment  by  the  group,  but  also 
reflect  the  degree  of  confidence  the  different  members  of  the  group  have  in  their 
estimates  (Levmore,  2003).  These  markets  provide  an  estimate  of  the  probability 
distribution  of  the  event  coming  true  via  the  range  of  trader  bets.  More  impressively, 
markets  using  market  scoring  rules^  aggregate  information  and  provide  a  joint  probability 
distribution  over  many  variables  by  allowing  bets  on  the  value  combinations  of  each 
dependent  variable  (Hanson,  2003).  In  other  words,  prediction  markets  using  a  market 
scoring  rule  mechanism  can  provide  probabilities  for  events  with  multiple  mutually 
exclusive  dependent  variables. 

Since  traders  participate  on  a  voluntary  basis  in  a  prediction  market,  those  who 
participate  tend  to  have  information  relevant  to  the  particular  prediction  (Abramowicz, 
2007).  Additionally,  traders  in  the  market  have  an  incentive  to  trade  at  their  earliest 
opportunity  upon  the  discovery  of  new  information  before  the  market  has  fully  priced  the 
new  information  (Abramowicz,  2007,  p.  17).  Confident  traders  have  the  ability  to 
speculate,  and  thus  move  the  trading  price  closer  to  the  actual  probability  of  the  event 
coming  true.  Therefore,  informed,  confident  traders  are  progressively  more  likely  to 
determine  trading  prices  (Abramowicz,  2007,  p.  7). 

The  mere  mention  of  market  speculation  naturally  leads  to  a  discussion  of 
manipulation.  Subsequent  chapters  more  fully  discuss  the  risk  of  manipulation. 
Additionally,  those  chapters  will  present  empirical  evidence  that  dismisses  the  effect  of 
manipulation  on  prediction  market  accuracy.  However,  the  next  paragraph  discusses 
manipulation  from  a  theoretical  perspective. 

Prediction  market  manipulation  is  a  potential  danger,  just  as  it  is  in  financial 
markets.  However,  the  motivation  for  traders  to  manipulate  is  not  nearly  as  high  as  it  is  in 
financial  markets,  especially  if  the  prediction  market  uses  play  money  as  currency.  The 

^  Market  scoring  rules  are  discussed  further  in  Chapter  V. 
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key  is  that  if  traders  are  aware,  in  advance,  that  other  traders  have  incentives  to 
manipulate,  they  will  counteract  their  manipulation  attempts  by  seeking  to  push  prices  in 
the  opposite  direction.  From  a  market  efficiency  perspective,  a  greater  incentive  exists  for 
higher  participation  by  informed  traders  in  markets  in  which  traders  are  not  trading 
according  to  market  fundamentals  (Abramowicz,  2007,  p.  31).  The  informed  traders  can 
take  advantage  of  the  uninformed  traders  for  an  easy  profit.  The  upside  to  the  market  is 
that  the  additional  insight  by  the  informed  market  entrants  should  increase  market 
accuracy.  Thus,  attempts  to  manipulate  the  market  may  actually  increase  market  accuracy 
(Hanson  &  Oprea,  2009,  p.  304). 

Whereas  insider  trading  is  illegal  in  the  financial  markets,  it  is  encouraged  in 
prediction  markets.  Thus,  decision  makers  are  afforded  access  to  information  possessed 
by  employees  with  “hidden  profiles” — those  who  would  otherwise  not  reveal  their 
information  because  of  the  nature  of  their  personality  or  hierarchical  constraints  (Dye, 
2008,  p.89). 

In  summary,  the  trading  prices  provide  decision  makers  with  a  continuously 
updated  picture  on  the  likelihood  of  future  events.  This  enables  decision  makers  to 
evaluate  risk  and  make  informed  resource  allocation  decisions.  Ultimately,  the  power  of 
prediction  markets  is  that  they  provide  incentives  to  the  traders  who  discover  new 
information  and  truthfully  reveal  it  while  the  market  provides  a  mechanism  for 
aggregating  trader  opinions  (Wolfers  &  Zitzewitz,  2004). 

E.  APPLICATIONS 

I.  Past  and  Current  Applications 

The  first  electronic  application  of  prediction  markets  was  the  Iowa  Presidential 
Stock  Market.  In  1988,  University  of  Iowa  economists  designed  this  market  to  predict  the 
outcome  of  U.S.  presidential  elections.  In  1992,  these  same  economists  presented  their 
first  academic  findings  that  suggested  these  markets  were  a  good  information  aggregation 
mechanism  (Forsythe,  Nelson,  Neumann,  &  Wright,  1992).  These  early  studies  laid  the 
foundation  for  prediction  markets  to  be  used  by  others  as  an  information  aggregation 
mechanism. 
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The  Defense  Advanced  Research  Projects  Agency  (DARPA)  was  involved  with, 
perhaps,  the  most  publicized  government  prediction  market  experiment  to  date.  In  July 
2003,  the  Futures  Markets  Applied  to  Prediction  (FutureMap)  program  was  intended  to 
explore  the  power  of  prediction  markets  to  aggregate  information  to  thwart  future 
terrorist  attacks.  Politicians  and  journalists  opposed  the  FutureMap  program  idea  in  the 
media.  They  publicly  attacked  DARPA  on  the  grounds  that  it  was  unethical  and  in  bad 
taste  to  place  wagers  on  the  fate  of  foreign  leaders  and  the  likelihood  of  terrorist  attacks 
(Looney,  2003).  Thus,  DARPA  cancelled  the  FutureMap  program  on  July  29,  2003,  the 
day  after  it  was  announced  (DARPA,  2003,  July  29). 

The  FutureMap  program  never  got  the  Policy  Analysis  Market  (PAM)  out  of  the 
planning  phase.  Thus,  it  was  never  operational.  However,  at  least  one  analysis  has  found 
that  it  would  have  been  self-defeating  (Richey,  2005).  The  argument  is  that  PAM  could 
be  effective  if  it  only  attempted  to  predict  terrorist  attacks.  However,  the  predictions 
surely  would  have  been  acted  upon  to  attempt  to  prevent  the  predicted  terrorist  attacks. 
This  prevention  action  would  subsequently  reduce  the  chance  of  the  event  occurring.  This 
would  be  an  excellent  collective  outcome  if  it  indeed  led  to  the  prevention  of  a  terrorist 
attack;  however,  it  would  undermine  the  self-interested  traders’  attempts  toward 
individual  profit  by  reducing  the  chances  of  the  event  occurrence  (Richey,  2005).  Applied 
to  other  prediction  markets,  this  phenomenon  can  also  exist  if  a  decision-maker  changes 
course  based  upon  the  prediction  market  forecast. 

Some  businesses,  especially  in  the  technology  and  pharmaceutical  industries, 
have  had  a  high  degree  of  success  employing  prediction  markets.  These  companies  have 
used  internal  prediction  markets  to  forecast  quarterly  sales  with  better  accuracy  than  their 
forecasting  departments.  Google  has  used  prediction  markets  to  determine  how  many 
people  will  use  their  applications,  such  as  Gmail  (Dye,  2008,  p.  87).  They  have  been  able 
to  make  resource  allocation  decisions  such  as  server  capacity  to  support  this  number  of 
users.  Similarly,  pharmaceutical  companies  have  used  prediction  markets  to  determine 
which  drugs  in  their  experimental  process  have  the  best  chance  for  approval  by  the 
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Food  and  Drug  Administration  (FDA).  Thus,  they  are  able  to  make  resource  allocation 
decisions  based  on  those  drugs  that  have  the  highest  probability  of  successfully  making  it 
to  market. 

Table  1  details  client  organizations,  the  type  of  prediction  market  they  are  using 
and  their  purpose  from  one  prediction  market  provider,  namely  Spigit.  Businesses 
employ  prediction  markets  to  aid  their  forecasting.  Some  examples  are  product  delivery 
dates,  product  uptake  rates,  manufacturing  capacity  needs,  product  ideas,  marketing 
campaigns  and  competitive  actions.  Similarly,  the  government  and  its  agencies  may  be 
able  to  use  prediction  markets  for  public  policy  and  decision-making. 


Table  1.  Spigit  Prediction  Market  Use  Cases  (From  Spigit,  n.d.) 


Company 

Market  Type 

Target  Commuruty 

Purpose 

Top  10  Pharmaceutical 

Restricted  betting 

A  community  of  biochemists,  managers,  and 

Predict  effectiveness  of  cartdidate 

Company 

Market 

IT  professionals 

compounds  for  treatment,  likelihood  i 
and  timing  of  F  DA  drug  approval, 
completion  date  of  internal  I 

development  pro)ects  j 

Ma^or  European  bank 

Continuous  Double 

Auctvon  Idea  Market 

All  bank  employees  mcludrnf  managers.  IT, 
customer-facmg  agents 

Determine  likelihood  of  an  idea 
succeeding  as  a  firiancial  product  or 
service 

Fortune  SOO 

Idea  Market  with 

Started  with  managers  and  senior 

Decision  aid  in  selecting  ideas  for 

Telecommunication  Company 

Market  Maker  Trading 

technofogists;  soon  to  expand  to  al  rank* 
and-file  employees 

Immediate  "anger  investment 

Ma>or  Insurance  Company 

Prediction  Market  with 
Market  Maker  Trading 

Invitation  only  community  of  thought 
leaders 

Predict  business  and  consumer  trends 
affecting  success  of  ideas  proposed  m 
the  idea  management  portal 

Fortune  SOO  Retail 

Restricted  Betting 
Market 

Multiple  user  groups  Including  employees, 
vendors,  customers  depef>dinc  on  the 
usage 

Applied  mainly  in  the  product 
marketing  area  to  gauge  product 
demand,  packaging  prefererKes.  etc.  | 

Fortune  500  Tech 

Prediction  Market  with 
Market  Maker  Trading 

international  commuruty  of  employees, 
partners,  ver^rs.  and  customers 

Used  in  conjunction  with  ideation 
event  to  assess  potential  of  selected 
ideas  in  terms  of  commerciabtation, 
partner  adoption,  etc 

2.  Future  Applicability 

Organizations  looking  to  improve  their  forecasting  of  contentious  issues  are  likely 
to  benefit  from  prediction  markets.  The  prediction  market  acts  as  not  only  a  mechanism 
by  which  many  people  can  state  their  beliefs,  but  also  shows  how  strongly  people  feel 
about  their  particular  belief(s)  in  a  quantitative  fashion.  If  so  inclined,  the  other 
participants  are  able  to  adjust  their  opinions  based  upon  the  rationale  of  other  traders.  As 
this  cycle  continues,  participants  provide  a  comprehensive  rationale  for  the  disparate 
viewpoints  within  the  group. 


13 


Organizations  considering  using  prediction  markets  should  start  with  pilot 
programs  so  they  can  compare  the  results  with  traditional  forecasting  techniques.  A 
learning  curve  will  occur  as  organizations  determine  which  circumstances  and 
participants  are  most  likely  to  result  in  accurate  forecasts. 

Prediction  markets  are  unlikely  to  be  useful  when: 

•  Outcomes  have  a  high  degree  of  predictability 

•  There  is  not  any  dispersed  information  to  aggregate 

•  There  are  only  a  limited  number  of  knowledgeable  traders. 
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III.  BENEFITS  AND  LIMITATIONS 


Prediction  markets  are  powerful  and  efficient  forecasting  tools  that  organizations 
can  use  under  a  multitude  of  external  conditions.  Prediction  markets  carry  with  them 
several  key  benefits  and  potential  limitations,  which  organizations  must  fully  understand 
and  appreciate  to  utilize  prediction  markets  effectively.  This  chapter  examines  the  most 
critical  of  those  benefits  and  limitations  and  provides  greater  insight  and  supporting 
evidence  for  each  in  relation  to  use  within  a  Navy  context. 

A.  KEY  BENEFITS 

Prediction  markets  have  a  proven  record  of  accomplishment  in  the  corporate 
context.  Such  a  record  comes  only  from  years  of  effort,  challenge,  error,  perseverance, 
and  insight.  The  insight  resulting  from  previous  experimental  and  practical  applications 
is,  perhaps,  most  valuable  from  the  preceding  list.  It  is  this  insight  that  develops  and 
reinforces  understanding  in  the  ways  prediction  markets  ought  to  be  used.  Moreover,  it  is 
this  insight  that  develops  and  reinforces  understanding  of  the  various  strengths  and 
capabilities  of  prediction  markets.  The  following  three  sections  examine  and  reflect  upon 
the  key  benefits  of  prediction  markets:  their  dynamic  nature,  accuracy,  and  potential  for 
anonymous  revelations. ^ 

1.  Dynamic  Nature 

A  major  advantage  of  a  prediction  market  is  its  dynamic  nature.  Prediction 
markets  generally  are  capable  of  continually  aggregating  information,  keeping  market 
managers  well  informed  as  to  the  group's  fluctuating  collective  belief  on  the  probability 
of  a  given  event's  occurrence.  The  prime  advantage  to  a  prediction  market's  dynamic 
nature  lies  in  a  prediction  market's  ability  to  reflect  participants'  collective  beliefs  in  light 
of  external  environmental  impacts,  such  as  significant  fluctuations  in  the  public  stock 
markets,  the  likelihood  of  war,  key  political  elections,  changes  in  public  or  private  policy. 


^  This  is  the  act  of  revealing  or  extracting  information  while  protecting  participant  identities. 
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as  well  as  numerous  other  possibilities.  Tziralis  and  Tatsiopoulos  (2007)  captured  the 
spirit  of  this  notion  and  went  further  with  the  following  excerpt: 

In  contrast  to  traditional  approaches,  the  operation  of  the  prediction 
market  is  not  affected  at  all  by  possible  changes  in  types  and  sources  of 
information  or  even  the  number  of  inputs  or  participants.  Prediction 
markets  are  by  nature  able  to  transform  unlimited  amounts  of  timely  and 
locally  dispread  qualitative  information  into  accurate  quantitative  forecasts 
about  the  future,  (p.  257) 

The  dynamic  nature  also  enhances  efficiency.  As  Tziralis  and  Tatsiopoulos 
observe,  “Some  of  the  most  difficult  steps  in  a  typical  forecasting  application  are  to  mine, 
namely  to  collect,  merge  and  clean  relevant  data  from  human  experts”  (2007,  p.  256). 
With  that,  prediction  markets  are  an  excellent  mechanism  for  accomplishing  and 
overcoming  such  tasks.  Moreover,  some  evidence  suggests  that  because  of  their  dynamic 
nature,  prediction  markets  can  provide  increasingly  improved  forecasts  over  time.  This 
generally  is  indicative  of  the  information  improving  over  time;  in  other  words,  prediction 
market  participants  become  better  informed  and  more  adept  at  using  the  markets  and 
more  fully  embrace  their  capabilities.  Overall,  this  leads  to  improved  forecasting  over 
time.  In  2008,  McKinsey  Quarterly  convened  a  roundtable  to  discuss  key  issues  relating 
to  prediction  markets  and  their  use  in  the  corporate  environment.  Among  the  panelists 
was  Bo  Cowgill,  Google's  product  manager,  who  is  a  leading  individual  in  implementing 
and  managing  prediction  markets  within  Google.  Mr.  Cowgill  offered  the  following 
observation  that  supports  the  ideas  above: 

The  longer  you  work  at  Google  and  the  longer  you  trade  in  the  prediction 
markets,  [.  .  .]  the  likelier  you  are  to  have  a  successful  trading  record.  [.  .  .] 

The  market  as  a  whole  got  smarter  [.  .  .],  and  we  could  observe  a  steady 
improvement  over  time.  (Dye,  2008,  p.  88) 

2.  Accuracy 

Accuracy  is  a  principle  advantage  of  prediction  markets.  Prediction  markets  can 
quickly  pull  together  information  spread  across  many  people  and  places,  and  the  accuracy 
of  the  results  can  rival  most  any  alternative  forecasting  method  under  optimal  conditions. 
Tziralis  and  Tatsiopoulos  (2007)  affirm  this  belief: 
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The  forecast  accuracy  of  an  efficient  prediction  market  is,  under  the 
condition  of  efficiency,  optimum.  In  practice,  prediction  markets  usually 
tend  to  perform  at  least  as  well  as  the  single  best  individual,  without 
requiring  knowledge  of  who  that  individual  is  in  advance,  (p.  256) 

Not  only  can  prediction  markets  match  the  accuracy  of  experts  and  alternate 
forecasting  methods,  prediction  markets  are  capable  of  surpassing  these  measures  under 
the  right  conditions.  Furthermore,  prediction  markets  have  accurately  performed  across  a 
variety  of  dynamic  event  subjects:  from  political  polling  to  corporate  performance  goals, 
and  from  important  Wall  Street  news  to  the  latest  Hollywood  releases  (Servan-Schreiber, 
Wolfers,  Pennock,  &  Galebach,  2004,  p.  244).  In  supporting  this  idea,  Robin  Hanson 
(2003)  cites  several  specific  examples  that  illustrate  prediction  market  accuracy:  Iowa 
Electronic  Markets  (lEM)  forecasts  repeatedly  outperformed  opinion  polls  on  various 
U.S.  presidential  elections;  Hewlett-Packard  markets  bested  traditional  printer  sales 
forecasts  in  75%  of  the  observances;  and  prediction  markets  outperformed  four  of  five 
expert  columnists  in  selecting  the  2000  Oscar  winners  (p.  107). 

Aside  from  their  ability  to  improve  upon  alternate  forecasting  methods,  prediction 
markets  also  are  capable  of  accurately  incorporating  information  more  quickly  than  other 
methods.  Eor  instance,  one  might  assume  that  a  prediction  market  on  a  given  presidential 
election  would  follow  the  lead  of  traditional  polls.  However,  Eorsythe,  Nelson,  Neumann, 
and  Wright  (as  cited  in  Abramowicz,  2003,  p.  18)  contend  that  analysis  of  the  1988  lEM 
highlights  that  traders  conducted  trades  based  on  information  that  was  subsequently 
mirrored  in  the  polls.  In  other  words,  traders  accurately  predicted  changes  to  the 
candidates’  expected  voting  returns  based  on  information  aside  from  traditional  polls,  and 
the  “poll  results  did  not  drive  market  prices”  (Abramowicz,  2003,  p.  18).^  This  supports 
the  idea  that  prediction  markets  are  capable  of  aggregating  given  information  more 
quickly  than  some  alternative  methods,  ceteris  paribus.  Although  the  lEM  case  results 
above  are  not  conclusive,  they  are  intriguing,  and  they  do  offer  supportive  evidence  on 
the  performance  capability  of  prediction  markets. 


6  In  fairness,  Abramowicz  (2003)  contrasts  this  point  by  citing  a  similar  election  study  in  the 
Netherlands  that  did  not  reach  the  above  conclusion  (pp.  18-19). 
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All  told,  the  accuracy  of  prediction  markets  may  not  be  superior  to  alternative 
forecasting  methods.  However,  numerous  studies  and  experiments  repeatedly  have  shown 
prediction  market  accuracy  at  least  approximates  that  of  alternate  methods.  For  instance, 
Wolfers  and  Zitzewitz  (2004)  analyzed  data  from  a  TradeSports  contract  that  paid  out 
only  if  Saddam  Hussein  was  removed  from  power  by  the  end  of  June  2003;  their  analysis 
shows  that  the  market  price  directly  corresponded  to  both  fluctuating  oil  prices  and  an 
expert  journalist’s  estimate  of  the  likelihood  of  a  U.S.-Iraq  military  conflict  (pp.  112- 
113). 

Accuracy  can  suffer  greatly  in  some  cases,  such  as  when  participation  is  minimal, 
or  when  information  is  not  readily  available  or  easily  interpreted  by  participants.  In  such 
instances,  managers  should  view  prediction  market  results  with  a  skeptical  eye.  However, 
under  desirable  market  conditions,  prediction  market  accuracy  likely  will  approximate 
that  of  alternative  forecasting  methods;  thus,  under  such  conditions,  accuracy  may  not  be 
the  ultimate  determinant  of  whether  an  organization  should  utilize  prediction  markets. 
Rather,  the  nature  and  amount  of  information  desired,  as  well  as  organizational  objectives 
may  have  a  more  prominent  role  in  such  a  decision.  Nonetheless,  if  various  forecasting 
methods  are  considered  for  a  given  objective,  an  organization  should  closely  examine  the 
motivation,  skill,  and  efficiency  of  its  current  forecasting  experts.  Generally,  these 
experts  will  provide  accurate  and  useful  information.  However,  Abramowicz  (2003) 
asserts  there  are  times  to  believe  prediction  markets  may  provide  greater  efficiency  or 
objectivity  than  expert  forecasters:  "It  is  in  governmental  decisionmaking  [5/c],  however, 
where  there  is  the  greatest  reason  to  be  suspicious  of  experts,  either  because  of  external 
influence  or  because  of  ideological  agendas”  (pp.  20-21).  For  this  reason,  and  others 
discussed  throughout  this  thesis,  prediction  markets  can  befit  government. 

3.  Anonymous  Revelations 

Another  key  benefit  to  prediction  markets  is  their  ability  to  aggregate  information 
from,  perhaps,  otherwise  unwilling  participants.  Often,  employees  find  themselves 
choosing  to  withhold  information  about  various  aspects  of  their  work.  This  withholding 
of  information  can  be  for  any  number  of  reasons,  such  as  protecting  their  reputation. 
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character,  or  job  out  of  fear  of  what  a  superior  might  do  with  the  information.  Moreover, 
some  employees  may  withhold  information  to  maintain  an  element  of  expert  power  or 
control.  From  an  organizational  behavior  standpoint,  such  control  may  serve  as  leverage 
in  maintaining  status  or  exchanging  reciprocal  benefits  among  co-workers.  Beyond  this 
point,  some  employees  act  in  a  role  that  keeps  them  relatively  hidden  in  terms  of 
expressing  opinions  or  beliefs  on  an  organization's  operations.  Todd  Henderson^ 
describes  this  in  the  following  way,  “.  .  .  information  can  reside  in  .  .  .  what  academics 
call  ‘hidden  profiles.’  These  are  people  within  an  organization,  who  because  of  their 
personality  or  position  in  the  hierarchy,  won’t  have  the  incentive  or  wherewithal  to  reveal 
information”  (Dye,  2008,  p.  89). 

Prediction  markets  are  an  excellent  means  of  bringing  out  sensitive  knowledge 
and  opinions.  Through  anonymity,  prediction  markets  allow  any  employee  to  have  an 
equal  say  by  enabling  hidden  employees  and  shielding  fearful  or  hesitant  employees. 
Prediction  markets  can  foster  a  spirit  of  participation  by  motivating  employees  to 
discover  and  share  information,  while  breaking  down  personal  or  political  organizational 
barriers  (Tziralis  &  Tatsiopoulos,  2007,  p.  257). 

B.  POTENTIAL  LIMITATIONS 

Because  of  years  of  hard  work  and  research  in  this  field,  prediction  markets  are 
now  better  understood  and  their  use  is  more  reality  than  possibility.  However,  prediction 
markets  are  not  without  weaknesses.  Just  as  experimental  and  practical  application  led  to 
insights  regarding  the  strong  suits  of  prediction  markets,  such  applications  also  led  to 
insights  regarding  the  shortcomings  of  prediction  markets.  The  following  four  sections 
identify  and  expound  upon  the  key  potential  limitations  of  prediction  markets:  issues 
associated  with  participation,  manipulation,  biases,  and  legal  restrictions. 


^  Todd  Henderson  served  as  a  panelist  on  the  McKinsey  prediction  market  roundtable.  At  the  time,  he 
was  an  assistant  professor  of  law  at  the  University  of  Chicago  Law  School,  with  a  background  in  law  and 
economics. 
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1.  Participation 

Active  and  productive  participation  is  an  essential  element  of  prediction  markets. 
Without  active  participation,  a  prediction  market  cannot  realize  its  full  potential,  and 
organizations  cannot  tap  into  the  associated  benefits  of  prediction  markets.  Participation 
heavily  hinges  on  potential  participants’  intrinsic  interest  or  desire  to  participate, 
incentives  used  to  elicit  such  participation,  and  even  on  participants’  understanding  of  the 
prediction  market  and  its  associated  questions  of  concern.  According  to  James 
Surowiecki,  “One  shortcoming  is  that  a  lot  of  people  inside  organizations  don’t  find  the 
market  mechanism  intuitive  or  easily  understood.  They  find  it  very  challenging  to  use, 
which  limits  the  pool  of  people  who  participate”  (Dye,  2008,  p.  89).  This  can  lead  to  a 
problem  with  attracting  a  diverse  participant  pool,  and  more  specifically,  a  problem  with 
attracting  uninformed  traders.^  Additionally,  prediction  markets  are  susceptible  to  an 
irrational  participation  problem  in  which  rational  traders  have  no  further  incentive  or 
desire  to  trade  once  they  already  have  hedged  their  bets  (Hanson,  2003,  p.  108).  For  these 
reasons,  and  others,  prediction  markets  may  become  thin.  The  following  sections  discuss 
problems  associated  with  thin  markets  and  the  inability  to  attract  uninformed  traders,  as 
well  the  importance  of  participant  training. 

a.  Thin  Markets 

Prediction  markets  require  a  sufficiently  large  and  continuously  active 
participant  pool.  The  greater  the  number  of  traders  and  the  more  diverse  the  participant 
pool,  the  more  likely  a  given  prediction  market  will  efficiently  and  accurately  aggregate 
information.  Moreover,  such  conditions  generally  are  necessary  to  incite  participants  to 
discover  or  reveal  new  information  pertinent  to  the  market.  Thin  markets  exist  when  there 
are  few  participants  and/or  participants  do  not  actively  participate  in  such  a  way  to  ensure 
continuous  trading  occurs.  This  does  not  suggest,  however,  that  a  market  that  reaches 
equilibrium  is  a  thin  market.  Rather,  a  thin  market  is  one  that  generally  stagnates  without 
reaching  equilibrium. 

^  Uninformed  traders  are  those  traders  who  do  not  possess  legitimate  or  relevant  information  with 
respect  to  a  given  market.  Uninformed  traders  are  necessary  to  assure  market  liquidity.  A  more  thorough 
discussion  follows  on  page  22. 
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Thin  markets  can  occur  because  of  poor  planning  or  minimal  management 
support.  Organizations  are  at  risk  of  dooming  any  given  prediction  market  from  its 
inception  when  they  implement  a  market  without  properly  planning  for  the  right  number 
and  demographic  of  respective  participants.  Thin  markets  also  can  result  from  market 
managers  introducing  too  many  questions  into  the  marketplace,  thereby  allowing 
participants  to  dilute  their  limited  trading  resources.  Additionally,  when  market  managers 
introduce  questions  that  require  a  high  degree  of  specialization  for  forecasting  purposes, 
such  questions  may  attract  a  very  limited  number  of  participants.  With  limited 
participation,  the  last  trade  on  a  given  prediction  market  likely  will  not  be  representative 
of  the  participants’  collective  belief  on  the  probability  of  a  given  event’s  occurrence.  This 
is  possible  because  relevant  information  may  be  concentrated  in  only  a  few  participants 
or  in  traders  who  already  have  hedged  their  bets  in  other  markets  (Abramowicz,  2003,  p. 
24). 

A  thin  market  generally  leads  to  comparatively  large  price  fluctuations, 
and  it  may  not  reach  equilibrium — due  to  the  lack  of  trading  activity  within  the  market. 
Price  fluctuations  often  occur  because  of  relative  information  asymmetry  that  exists 
among  the  various  traders.  In  particular,  increasing  degrees  of  information  asymmetry  in 
double  auction  markets  will  lead  to  increased  bid-ask  spreads;  as  a  result,  participants 
will  conduct  trades  less  frequently  (Abramowicz,  2003,  p.  24).  Robin  Hanson  (2003) 
offers  the  following  example,  “Consider  the  case  where  a  single  person  knows  something 
about  an  event,  and  everyone  else  knows  that  they  know  nothing  about  that  event.  In  this 
case,  standard  information  markets  based  on  that  event  simply  cannot  acquire  this 
person’s  information”  (p.  108).  Due  to  the  price  fluctuations  in  thin  markets,  decision 
makers  should  not  blindly  trust  short-term  price  spikes.  Rather,  decision  makers  should 
utilize  a  price  smoothing  method — such  as  averaging  prices  of  a  set  number  of  days  prior 
to  the  market’s  closing — to  diminish  effects  of  extreme  price  fluctuations  prior  to  making 
subsequent  decisions  on  the  market’s  data  (Abramowicz,  2003,  p.  14). 

Although  there  are  no  straightforward  solutions  to  thin  markets,  Hanson 
(2003)  argues  that  market  scoring  rules  can  help  correct  problems  associated  with  thin 
markets.  Market  scoring  rules  use  a  mathematical  algorithm  to  avoid  dependence  upon 
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person-to-person  trading — the  algorithm  updates  market  prices  for  a  given  market  based 
on  that  market’s  trade  history.  Hanson’s  market  scoring  rules  are  discussed  more  fully  in 
Chapter  V.  Additionally,  Chapter  VI  reports  observed  aspects  of  Hanson’s  market 
scoring  rules,  as  they  act  as  a  basis  for  Inkling ’s^  prediction  market  platform. 

b.  Attracting  Uninformed  Traders 

As  mentioned  above,  prediction  markets  require  a  relatively  large  and 
diverse  group  of  active  traders.  Often,  a  critical  issue  with  prediction  markets  is  attracting 
both  informed  and  uninformed  traders.  Informed  traders  are  those  who  possess  legitimate 
knowledge — ^knowledge  that  is  not  based  purely  on  speculation  or  hearsay — with  respect 
to  a  given  prediction  market;  conversely,  uninformed  traders  are  those  who  do  not 
possess  such  knowledge.  Information  can  only  be  relayed  and  exchanged  within  a  given 
market  when  some  information  gap  exists.  This  existence  of  an  information  gap  heavily 
relies  on  the  presence  of  uninformed  marketplace  traders.  According  to  Wolfers  and 
Zitzewitz  (2006),  “the  success  of  the  prediction  market  in  generating  trade  depends 
critically  on  attracting  uninformed  traders”  (p.  7).  It  should  be  noted  that  although 
organizations  can  attempt  to  ascertain  or  predetermine  which  traders  are  informed  or 
uninformed,  this  cannot  be  simply  determined.  In  fact,  it  is  quite  improbable  that 
organizations  will  know  which  traders  are  informed  until  prediction  market  results  are 
available.  Nevertheless,  organizations  must  make  every  effort  to  attract  a  sizeable  and 
diverse  participation  pool,  specifically  to  include  uninformed  traders. 

To  attract  uninformed  traders,  organizations  must  understand  the 
underlying  motivations  of  potential  participants.  Generally,  the  pure  thrill  of  competition 
and  the  side  effect  of  personal  entertainment  is  enough  to  pique  some  potential 
participants’  interests.  Competitiveness,  overconfidence,  and  entertainment  are  intrinsic 
values  that  many  humans  share;  organizations  will  do  well  to  tap  into  these  natural 
sources  of  motivation  to  develop  an  efficient  and  liquid  market  (Wolfers  &  Zitzewitz, 
2006,  pp.  7-9). 


9  Our  experimental  pilot  marketplace  used  Inkling’s  platform.  This  is  discussed  more  fully  in  Chapter 
VI  of  this  thesis. 
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However,  incentives  are  likely  necessary  to  draw  in  a  fully  diverse  and 
sufficiently  large  body  of  participants.  Prediction  market  experts  support  this  point 
widely,  and  it  is  highlighted  in  Chapter  VII.  Organizations  must  appropriately  match 
incentives  to  the  participant  pool,  and  they  must  fully  consider  the  way(s)  in  which  they 
should  incentivize  participants.  For  instance,  if  organizations  only  reward  the  highest 
earner  for  a  given  period  or  market,  many  participants  may  stop  trading  once  they  no 
longer  have  a  chance  to  contend  for  the  reward.  Conversely,  some  participants  may  make 
large  investments  in  unlikely  outcomes  to  have  a  chance  at  becoming  the  highest 
earner — this  of  course  leads  to  market  inefficiency  and  affects  overall  market  accuracy. 
Beyond  considering  who  and  when  to  reward,  organizations  must  also  consider  the  types 
and  nature  of  rewards.  This  last  aspect  can  become  quite  challenging,  and  it  may  require 
a  great  deal  of  creativity  to  keep  the  marketplace  fresh,  fun,  and  interesting.  Jeff 
Severtsio,  Best  Buy’s  prediction  market  pundit,  shared  the  following  regarding  the 
challenges  of  attracting  and  incentivizing  participants: 

You  always  have  to  be  marketing  them  [prediction  markets],  just  like 
everything  else.  [.  .  .]  Every  quarter,  you  have  to  refresh  your  list  of  prizes 
and  try  to  come  up  with  something  at  least  as  compelling  as  the  last  time. 

(Dye,  2008,  p.  89) 

To  close,  the  participation  issue  cannot  be  overemphasized  here,  and  it 
certainly  cannot  be  overlooked  upon  implementation.  Participation  is  the  most  critical 
aspect  in  the  practical  application  of  prediction  markets.  Regardless  of  who  sponsors  a 
given  market  or  what  questions  managers  ask,  a  prediction  market  has  little  chance  of 
aggregating  information  effectively  if  appropriate  incentives  do  not  exist  or  the 
participant  pool  is  not  appropriately  diversified. 


Jeff  Severts  served  as  a  panelist  on  the  McKinsey  prediction  market  roundtable.  At  the  time,  he  was 
the  Vice  President  and  General  Manager  of  Best  Buy’s  Geek  Squad,  with  work  involving  forecasting 
models  and  prediction  markets. 
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c.  Participant  Training 

Participant  training  can  be  an  underappreciated  element  in  the  market 
implementation  process.  If  an  organization  does  not  properly  train  participants,  providing 
them  with  a  base  understanding  of  the  prediction  market  concept,  then  the  marketplace 
may  struggle  to  maintain  participants  in  the  pool.  Following  the  experimental  prediction 
market  conducted  in  association  with  this  thesis,  an  anonymous  post-market  survey  was 
conducted  to  poll  participants  on  the  implementation,  operation,  and  outcome  of  the 
experiment.  Multiple  survey  responses  indicated  that  participants  stopped  trading  beeause 
they  did  not  understand  the  markets.  This  likely  resulted  from  improper  participant 
training. 

Although  it  is  imperative  that  market  managers  attract  uninformed  traders, 
it  also  is  essential  that  at  least  the  majority  of  the  traders  understand  the  prediction  market 
concept.  Traders  must  realize  that  not  everyone  will  be  an  informed  trader  in  every 
market.  In  fact,  most  traders  will  not  be  fully  informed  in  most  markets.  It  is  neither 
realistic  nor  efficient  for  the  majority  of  participants  to  be  fully  informed,  as  the  market 
would  become  stagnant  without  information  asymmetry,  which  is  necessary  for  trading  to 
occur.  However,  each  participant  may  have  some  general  and  unique  information  with 
respeet  to  certain  markets.  Therefore,  participants  need  to  understand  that  in  every  market 
some  uninformed  traders  must  participate  to  ensure  market  efficiency,  even  if  this  means 
trading  only  on  inclinations  or  market  trends.  It  is  a  matter  of  knowing,  understanding 
and  embracing  the  various  roles  within  the  market’s  participation  pool,  as  well  as 
understanding  that  traders  can  act  in  various  roles  at  various  times. 

d.  Understanding  Trading  Strategies:  Short  selling 

To  further  enhance  market  efficiency,  the  majority  of  traders  must 
understand  the  ways  in  which  they  can  utilize  their  market  currency  within  a  given 
market.  More  specifically,  traders  must  understand  that  they  have  the  ability  to  trade  long 
or  short.  In  addition,  they  must  know  how  best  to  do  either,  based  on  given  market 
conditions  and  their  beliefs  on  a  given  event’s  likelihood  of  occurrence. 
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Trading  long  is  intuitive  and  relatively  easy  for  most  traders  to  understand 
and  embrace.  The  basic  gist  is  to  buy  low  and  sell  high,  or  hold  for  an  expected  positive 
payout  following  the  market’s  close.  Market  inefficiencies  can  occur  when  traders  do  not 
understand  or  embrace  their  opportunities  to  sell-short.  Short  selling  is  the  act  of  bidding 
against  (or  selling  shares  one  does  not  own  for)  an  event  contract  that  a  trader  believes 
has  an  over-valued  (or  too  high)  probability  of  occurrence.  It  effectively  is  the  reverse  of 
trading  long.  With  short  selling,  one  effectively  sells  high,  and  later  buys  low — the  twist 
is  the  seller  does  not  own  any  shares  at  the  time  of  the  sale,  but  rather  sells  on  credit  with 
a  promise  to  purchase  the  same  number  of  shares  at  a  later  time.  In  other  words,  if  most 
of  the  market  participants  believe  an  event  will  occur  and  a  few  traders  believe  that  the 
event  has  a  low  probability,  then  the  few  can  conduct  short-sales  with  an  expectation  that 
prices  ultimately  will  drop  or  move  to  zero  at  market  closing.  According  to  Brigitte 
Yuille,  short  selling  contributes  liquidity,  efficiency  and  a  voice  of  reason  in  bull  markets 
(Yuille,  n.d.). 

An  example  can  help  illustrate  the  process  and  its  advantages.  Assume  a 
given  market  price  is  $  0.80,  with  payoffs  set  at  $1.00  if  event  y  occurs  and  $0  if  event  y 
does  not  occur — one  can  infer  that  the  market  collectively  assesses  an  80  percent 
probability  of  event  y  occurring.  First,  consider  the  act  of  trading  long.  If  trader  Joe 
believes  event  y  will  occur,  he  may  purchase  one  share  at  $  0.80.  If  event  y  occurs,  trader 
Joe  is  paid  $1.00;  he  profits  $  0.20  ($1.00  payout  minus  his  $  .80  purchase  price).  Now 
assume  trader  Joe  does  not  believe  event  y  will  occur;  he  may  elect  to  short-sell  event  y. 
Trader  Joe  would  sell  one  share  at  $  .80,  with  a  promise  to  buyback  that  one  share  at  a 
later  time.  If  he  holds  the  share  and  market  price  decreases  to  $  .30,  he  may  buy  back  the 
one  share  at  $  0.30.  Thus,  trader  Joe  profits  $  0.50  ($  .80  revenue  from  the  initial  sale 
minus  $  0.30  from  the  buyback).  Moreover,  if  he  holds  the  share  until  the  market  closes 
and  event  y  does  not  occur,  he  will  buy  back  the  one  share  at  $0.  Thus,  trader  Joe  profits 
$  0.80  ($  .80  revenue  from  the  initial  sale  minus  $0  from  the  buyback).  However,  if  trader 
Joe  conducted  his  short-sale  as  above  and  event  y  does  occur,  he  would  lose  $  0.20  ($  .80 
revenue  from  initial  sale  minus  $1.00  from  the  buyback). 
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Buyback  can  occur  at  any  time  prior  to  market  closing;  thus,  the  figures 
above  are  purely  arbitrary  for  the  sake  of  illustrating  the  concept.  People  often  conduct 
their  buyback  prior  to  market  closing  either  to  collect  profit  from  a  decline  in  a  contract’s 
market  price  or  to  cut  losses  from  a  subsequent  rise  in  a  contract’s  market  price.  In  a 
short-sale,  the  risk  of  loss  is  much  the  same  as  that  associated  with  trading  long,  but  the 
risk  may  be  greater  in  terms  of  amount  per  share  when  one  sells-short  against  low 
probability  events. 

Risk  of  loss  is  not  the  only  drawback  to  short  selling.  When  a  trader  short- 
sells,  a  portion  of  that  trader’s  marketplace  assets  are  placed  into  holding  until  the 
subsequent  buyback  occurs,  and  these  funds  in  holding  cannot  be  used  for  any  other 
marketplace  security  until  the  buyback.  The  amount  held  in  reserve  is  equal  to  the 
maximum  cost  of  buying  back  every  shorted  share  under  the  worst-case  scenario — this  is 
roughly  equal  to  the  number  of  shares  shorted  multiplied  by  the  positive  payout  price. 
This  means  that  significant  portions  of  a  trader’s  market  assets  may  be  held  in  reserve  for 
high  volume  short-sales,  which  ties  up  valuable  assets  from  other  marketplace  ventures. 

Traders  must  understand  the  implication  of  short  selling,  as  it  can  greatly 
affect  the  way  in  which  they  choose  to  conduct  trades  with  their  personal  knowledge 
under  given  market  conditions.  Furthermore,  if  traders  do  not  understand  the  above 
implications,  the  market  as  a  whole  cannot  function  as  efficiently  as  possible.  New 
information  may  be  revealed  much  more  slowly,  or  not  at  all. 

Proper  training  can  help  establish  baselines  for  participants  with  respect  to 
efficient  trading  strategy.  Good  training  can  minimize  much  inefficiency  before  it  occurs, 
and  it  can  prevent  traders  from  becoming  frustrated  by  common  but  unintuitive 
marketplace  occurrences.  This  in  turn  helps  maintain  a  reliable  participation  pool. 

2.  Manipulation 

Manipulation  is  a  common  concern  when  considering  prediction  market 
utilization.  There  are  several  instances  of  known  attempts  to  manipulate  prediction 
markets,  though  these  attempts  at  market  manipulation  generally  have  failed.  However,  it 
is  conceivable  that  some  participants  may  attempt  to  manipulate  a  given  market.  For 
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instance,  some  unethical  traders  may  explicitly  and  intentionally  spread  false  information 
to  drive  down  the  price  of  a  given  contract  subsequent  to  short  selling  within  that  market 
(Yuille,  n.d.).  Furthermore,  traders  may  attempt  market  manipulation  to  personally  profit 
or  to  influence  resultant  decisions  linked  to  a  prediction  market.  However,  profit  motives 
for  all  participants  generally  ensure  such  manipulation  attempts  are  not  successful 
(Wolfers  &  Zitzewitz,  2004,  p.  119).  Since  all  traders  have  incentives  to  predict 
accurately,  any  erroneous  or  manipulative  trades  likely  will  be  counteracted  and  corrected 
by  other  knowledgeable  and  informed  traders. 

In  terms  of  manipulation  for  personal  profit,  Wolfers  and  Zitzewitz  (2006) 
highlight  several  examples  in  which  known  manipulation  attempts  resulted  in  only  short¬ 
term  price  fluctuations,  which  the  markets  subsequently  corrected  within  24  hours  (p. 
11).  However,  the  examples  cited  by  Wolfers  and  Zitzewitz  involved  relatively  thick 
markets.  The  extent  to  which  a  given  market  can  be  manipulated  depends  on  the  market’s 
thinness  (Wolfers  &  Zitzewitz,  2004,  p.  119).  With  a  thin  market,  the  ability  to  fend  off 
manipulation  attempts  resides  with  many  fewer  participants,  and  the  potential  for  market 
manipulation  is  greater. 

With  respect  to  manipulation  to  influence  resultant  decisions,  such  attempts  can 
involve  individuals  or  groups.  In  particular,  Michael  Abramowicz  discussed  the  notion  of 
unrepresentative  decision  makers  by  which  resultant  decisions  are  made  or  influenced 
(Abramowicz,  2003,  pp.  46-51).  Abramowicz  suggested  that  prediction  markets  that 
affect  resultant  decisions  could  have  a  participant  pool  that  lacks  diversity,  or  is 
unrepresentative  of  the  general  population  that  is  affected  by  the  resultant  decision(s).  In 
such  cases,  the  non-diverse  participant  pool  either  may  intentionally,  or  through  an 
unintended  social  judgment  bias^i,  systemically  provide  unrepresentative  inputs  into  the 
prediction  market  leading  to  a  particular  resultant  decision.  Again,  this  is  of  particular 
concern  with  thin  markets.  Decision  makers  must  be  aware  and  consider  such 
possibilities  when  utilizing  prediction  markets  and  establishing  associated  participation 
pools. 


See  page  29  for  more  on  social  judgment  bias. 
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3. 


Biases 


Biases  are  important  elements  to  consider  when  assessing  prediction  market 
accuracy  and  correlating  market  prices  to  event  probabilities.  Biases  affect  how  people 
interact,  perceive,  and  share  information.  This  strong  sense  of  perception  related  to 
critical  events  and  conditions  can  lead  people  to  conduct  trades  somewhat  irrationally. 
Moreover,  biases  may  cause  participants  to  repeatedly  over-  or  under-value  specific 
contracts.  The  following  sections  briefly  discuss  four  of  the  most  prevalent  biases  of 
which  decision  makers  should  be  aware:  1)  over-optimism,  2)  under-pricing  of  extremes, 
3)  long  shot,  and  4)  social  judgment. 

a.  Over-optimism 

The  over-optimism  bias  is  one  in  which  employees  hold  an  overly 
optimistic  view  of  their  organization  and/or  its  ability  to  achieve  certain  goals.  Most 
organizations  generally  would  appreciate  having  a  problem  such  as  this,  but  this  bias  can 
affect  prices  and  associated  probabilities  among  prediction  markets  related  to  an 
organization’s  goals  or  performance.  Bo  Cowgill  noticed  this  bias  in  Google’s  prediction 
markets: 

The  cause  seems  to  be  new  employees,  whose  trades  show  that  they  are 
highly  optimistic  about  our  company.  The  external  Google  stock  price  also 
seems  to  play  a  role.  [.  .  .]  People  feel  excited  about  the  company  when  the 
stock  performs  well,  so  they’re  more  likely  to  bet  that  good  things  will 
happen  to  Google.  (Dye,  2008,  p.  88) 

b.  Under-pricing  Extremes 

Under-pricing  extremes  is  a  common  occurrence  in  which,  generally,  risk- 
averse  participants  underestimate  the  likelihood  of  low -probability  events.  This  may  be 
especially  prevalent  when  a  market  has  contracts  with  a  relatively  high  actual  probability 
of  occurrence;  by  default,  such  markets  will  have  some  contracts  with  relatively  low 
actual  probabilities  of  occurrence.  In  this  case,  traders  may  under-price  the  low 
probability  event(s)  by  trading  more  heavily  in  favor  of  the  high  probability  event(s).  In 
effect,  traders  may  assume  away  the  possibility  of  a  low -probability  event  actually 
occurring.  For  instance,  Google  observed  this  bias  when  utilizing  prediction  markets  to 
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forecast  the  number  of  people  who  would  use  a  given  service:  “When  we  floated 
contracts  with  five  different  outcomes — for  example,  forecasts  about  the  number  of 
Gmail  users — the  highest  and  the  lowest  outcome  happened  more  often  than  the  market 
expected”  (Dye,  2008,  p.  88). 

c.  Long  Shot 

The  long  shot  bias  is  nearly  the  opposite  of  under-pricing  extremes.  Many 
risk-seeking  people  will  overestimate  the  probability  of  low-probability  events  occurring. 
This  can  also  lead  to  price  or  probability  distortion  as  high-probability  events  have  prices 
that  reflect  lower-than-actual  probabilities;  conversely,  low-probability  events  have 
prices  that  reflect  higher- than-actual  probabilities.  The  effects  of  long  shot  bias  and 
under-pricing  extremes  might  counterbalance  one  another  in  markets  characterized  by 
both  biases,  although  to  our  knowledge  no  empirical  evidence  exists  to  support  this. 

d.  Social  Judgment 

Social  judgment  is  based  loosely  on  social  judgment  theory;  to  our 
knowledge,  it  is  not  a  supported  theory  or  bias  among  economists  or  prediction  market 
experts.  Rather,  social  judgment  is  used  here  as  a  way  of  categorizing  and  labeling  all 
personally  and  professionally  held  biases  that  are  based  on  one’s  prior  experiences, 
current  circumstances,  and  any  associated  social  networks  (whether  formal  or  informal  in 
nature).  The  principle  idea  is  that  all  people  hold  biases  that  are  continuously  formed  and 
reshaped  by  socio-psychological  experiences  over  the  course  of  one’s  life.  In  general, 
decision  makers  may  assume  that  such  biases  are  universal,  unavoidable,  and  have  no  net 
effect  on  a  prediction  market.  However,  we  hypothesize  that  in  some  instances  this  bias 
could  affect  a  given  market. 

To  understand  the  way  in  which  social  judgment  may  affect  a  given 

market,  one  first  must  understand  the  ways  in  which  such  biases  are  reinforced.  People 

tend  to  socially  network  in  unique  and  recurring  ways;  when  people  socially  network 

with  others  who  possess  common  social  biases,  the  social  judgments  of  the  group  may 

become  reinforced  or  strengthened.  This  strengthening  of  judgments  likely  depends  on 

the  size  of  the  social  group  involved,  as  well  as  the  strength  of  respective  individual 
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biases  among  the  group.  Depending  on  a  given  organization’s  size  and  makeup,  as  well 
as  the  nature  and  amount  of  information  sharing,  the  group  strengthening  of  judgments 
could  potentially  skew  prediction  market  probabilities  for  a  given  market.  In  short,  the 
chief  concern  for  this  bias  arises  when  a  prediction  market  operates  with  a  participant 
pool  that,  unintentionally  or  not,  is  homogeneous.  Such  a  group  likely  would  share 
subjective  opinions  and  beliefs  across  a  number  of  topics. 

Interestingly,  a  benefit  may  be  gained  from  the  social  judgment  bias. 
Again,  depending  on  a  given  organization’s  size  and  makeup,  as  well  as  the  nature  and 
amount  of  information  sharing,  this  bias  may  serve  to  highlight  the  ways  in  which 
information  is  gathered  and  shared  within  a  given  organization.  This  may  offer  insight  to 
an  organization’s  formal  and  informal  social  and  communication  networks.  Google 
recognized  observable  network  communication  patterns  among  its  employees  during 
prediction  market  trading: 

Our  [Google’s]  markets  showed  that  beliefs  are  clustered,  and  these 
clusters  are  made  up  of  individuals  who  physically  sit  and  work  close  to 
each  other.  .  .  .  Clusters  also  form  around  working  together,  socializing 
outside  of  work,  and  speaking  a  common  language,  even  when  this 
doesn’t  involve  sitting  close  by.  (Dye,  2008,  p.  88) 

4.  Legal  Restrictions 

Legal  restrictions  may  present  one  of  the  chief  limitations  of  prediction  markets. 
In  particular,  anti-gambling  laws  are  the  principal  barrier  to  real  money  prediction 
markets  (Hanson,  2003,  p.  107).  Moreover,  Arrow  et  al.  (2007)  claim,  “Current  laws  and 
regulations  affecting  the  use  of  prediction  markets  in  the  United  States  are  likely  to 
stymie  innovation,  and  thus  reduce  economic  welfare”  (p.  2). 

However,  Arrow  et  al.  (2007)  suggest  three  ways  an  organization  may  reduce  its 
legal  risk  with  respect  to  real  money  prediction  markets:  1)  obtain  a  no-action  letter  from 
the  Commodity  Futures  Trading  Commission  (CFTC),  2)  provide  allotments  to 
participants  so  they  do  not  risk  losing  their  own  money,  and  3)  listing  the  prediction 
market  on  a  traditional  futures  exchange  (p.  2).  Interestingly,  Arrow  et  al.  also  observe 
that  while  the  CFTC  oversees  several  prediction  markets  at  the  federal  level,  only  the 
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Iowa  Electronic  Markets  operates  with  real  money  because  “its  [lEM]  researchers  were 
able  to  obtain  a  letter  from  the  CETC  that  permitted  them  to  do  so  under  certain  limited 
conditions”  (p.  2). 

This  chapter  has  highlighted  the  key  benefits  and  limitations  associated  with 
prediction  markets.  The  following  are  the  key  benefits  of  prediction  markets:  their 
dynamic  nature,  accuracy,  and  potential  for  anonymous  revelations.  Prediction  markets 
are  dynamic  in  nature;  that  is,  they  are  capable  of  efficiently  aggregating  information  in  a 
continuous  or  ongoing  manner.  Additionally,  prediction  markets  are  capable  of 
performing  their  information  aggregation  with  a  high  degree  of  accuracy;  they  can 
quickly  pull  together  information  spread  across  many  people  and  places,  and  the  accuracy 
of  the  results  can  rival  most  any  alternative  forecasting  method  under  optimal  conditions. 
Moreover,  prediction  markets  allow  users  to  express  personal  opinions  and  beliefs  on 
specific  topics  in  an  anonymous  fashion;  this  encourages  feedback  at  all  organizational 
levels  without  any  fear  of  management  or  peer  reprisal. 

The  chief  limitations  addressed  in  this  chapter  are  those  associated  with 
participation,  manipulation,  biases,  and  legal  restrictions.  Prediction  markets  require 
some  minimum  level  of  participation  to  avoid  stagnation;  without  enough  participants  in 
the  active  pool,  active  traders  may  lose  interest  and/or  incentive  to  share  their  information 
with  other  active  traders.  Additionally,  manipulation  can  become  a  viable  concern  in 
prediction  markets  that  have  little  active  participation.  This  chapter  also  discussed  the 
following  biases  that  can  affect  trading,  and  thus  accuracy:  over-optimism,  under-pricing 
extremes,  long  shot,  and  social  judgment.  Einally,  legal  restrictions  can  limit  some 
organizations’  ability  to  easily  or  efficiently  implement  and  manage  prediction  markets; 
legal  restrictions  generally  have  the  greatest  impact  on  those  who  seek  to  utilize  real 
money  prediction  markets,  as  government  concerns  with  gambling  and  ethics  move  to  the 
forefront. 
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IV.  WHEN  TO  USE  PREDICTION  MARKETS 


Prediction  markets  can  supersede  geographic  boundaries  and  cultural  barriers  by 
quickly  aggregating  information,  and  they  can  do  so  with  high  quality.  However,  there 
can  be  considerable  cost  and  effort  involved  in  establishing  these  markets.  According  to 
Tziralis  and  Tatsiopoulos  (2007),  it  is  essential  that  organizations  use  prediction  markets 
when  the  desired  forecast  importance  is  relatively  high,  to  optimize  organizational  efforts 
and  spending  involved  in  implementing,  and  maintaining  market(s)  (p.  257).  One  of  the 
most  critical  questions  to  consider  is  that  of  when  to  use  prediction  markets.  Several 
factors  have  an  important  role  in  determining  the  answer  to  this  key  question,  including 
time-sensitivity  of  the  desired  information,  importance  of  information  and  forecasts, 
required  quality,  and  cost  or  budget  constraints.  For  simplicity,  these  factors  can  be 
grouped  into  two  broad  categories:  1)  forecast  objectives  and  2)  market  feasibility.  The 
following  sections  discuss  various  considerations  of  each  category  with  respect  to 
determining  whether  an  organization  should  use  a  prediction  market. 

A.  FORECAST  OBJECTIVES 

Forecasting  objectives  can  assist  in  determining  the  forecasting  importance,  and 
they  can  aide  in  selecting  the  proper  forecasting  means  for  the  organization’s  desired 
ends.  Organizations  should  use  statistical  and  econometric  methods  when  there  is 
sufficient  historical  data  and  information  to  support  forecasting  (Chen,  2005).  Such 
methods  carry  cost  benefits  as  well  as  familiarity  and  ease  of  utilization.  However,  such 
methods  generally  cannot  match  the  detail,  flexibility,  and  efficiency  that  prediction 
markets  offer. 

Prediction  markets  are  superior  to  other  forecasting  methods  in  their  ability  to 
aggregate  dispersed  information  efficiently.  The  following  excerpt  from  Yiling  Chen’s 
doctoral  thesis  (2005)  supports  this  conclusion: 

Information  markets  are  more  suitable  when  information  about  future 
events  is  dispersed  among  an  organization  or  society,  especially  when 
information  only  exists  as  tacit  knowledge  or  those  who  have  information 
tend  to  not  reveal  it.  (p.  120) 
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Additionally,  a  McKinsey  conference  roundtable  discussion  (Dye,  2008)  on 
prediction  markets  provides  an  example  that  illustrates  these  sentiments.  Among  the 
panelists  was  Jeff  Severts  of  Best  Buy.  He  offered  multiple  accounts  in  which  ground 
level  employee -based  prediction  markets  bested  the  conventional  forecasting  techniques 
used  by  his  professional  forecasting  experts  for  forecasting  holiday  gift  card  sales  (Dye, 
2008). 

The  nature  of  forecast  objectives  likely  will  not  determine  exclusively  whether  to 
use  prediction  markets  as  a  primary  forecasting  tool.  Rather,  the  obstacles  presented 
through  market  use  also  will  play  an  important  role.  In  spite  of  their  strong  benefits, 
prediction  markets  certainly  carry  unique  limitations.  Prediction  markets  require  a  useful 
body  of  pertinent  knowledge  in  addition  to  an  assembly  of  users  who  continually 
participate  with  an  accurate  perception  of  their  collective  knowledge  pool.  Wolfers  and 
Zitzewitz  (2004)  offer  a  strong  assessment  of  these  two  requirements: 

As  such,  these  markets  are  unlikely  to  perform  well  when  there  is  little 
useful  intelligence  to  aggregate  or  when  public  information  is  selective, 
inaccurate  or  misleading.  [...]  For  example,  the  public  information  on  the 
probability  of  weapons  of  mass  destruction  in  Iraq  appears  to  have  been  of 
dubious  quality,  so  it  is  perhaps  unsurprising  that  [...]  the  markets  were  as 
susceptible  as  general  public  opinion  to  being  misled,  (pp.  121-122) 

B.  MARKET  FEASIBILITY 

In  determining  a  market’s  feasibility,  an  organization  first  must  fully  understand 
its  desired  market  objectives.  The  desired  results  should  heavily  affect  the  market’s 
format  through  specific  and  clear  market  contracts  (posed  questions).  Broad  or  unclear 
questions  will  attract  less  meaningful  information  inputs  to  the  market.  When 
determining  the  feasibility  of  prediction  markets,  organizations  must  consider  three  key 
issues: 

•  Nature  of  available  information 

•  Cost 

•  Political  and  Legal  Considerations 

An  organization  must  assess  the  nature  of  available  information.  To  make  a 

prediction  market  worthwhile,  information  must  be  relevant  and  available  for 
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aggregation.  Additionally,  the  number  of  questions  ean  drastieally  affeet  a  market’s 
performance.  Therefore,  an  organization  should  not  use  too  few  or  too  many  questions  to 
aggregate  information  from  market  participants.  If  there  are  too  few  questions,  an 
organization  cannot  aggregate  information  effectively  for  optimal  forecasting.  If  there  are 
too  many  questions,  market  liquidity  will  decrease,  and,  in  addition,  the  resource  or  time- 
cost  for  managing  the  overall  market  will  increase.  Moreover,  cost  is  generally  the 
deciding  factor  when  organizations  assess  desirability  of  taking  on  new  investments,  so 
increased  costs  may  deter  or  limit  organizations  from  using  prediction  markets.  Finally, 
certain  political  or  legal  restrictions  may  present  challenges  or  obstacles  to  organizations 
implementing  certain  prediction  markets.  The  following  sections  more  fully  discuss  each 
of  the  aspects  in  turn. 

1.  Nature  of  Available  Information 

A  prediction  market’s  quality  is  no  better  than  the  sum  of  the  available 
information  it  seeks  to  aggregate.  Furthermore,  a  market  relies  on  informed  and 
uninformed  participants  to  sustain  market  liquidity  and  accuracy.  Therefore,  an 
organization  should  only  use  a  prediction  market  when  it  expects  that  the  dispersed 
information  is  relevant  and  of  high  quality.  Moreover,  an  organization  must  ensure  it  can 
properly  incentivize  sufficient  participants  to  aggregate  the  desired  information 
effectively.  “Whether  there  are  people  with  relevant  information,  and  whether  these 
people  can  be  attracted  to  participate  are  important  for  better  predictions”  (Chen,  2005,  p. 
123).  Thus,  failing  to  meet  either  condition  above  should  dissuade  an  organization  from 
developing  a  prediction  market  as  its  primary  forecasting  tool. 

2.  Cost 

Cost  is  a  critical  issue  associated  with  a  greater  number  of  questions.  With  more 
market  questions,  the  total  cost  will  be  higher  in  the  form  of  additional  time  and  resources 
necessary  for  operating  and  maintaining  the  market.  Moreover,  significant  time  and 
resources  are  required  to  plan,  design,  and  implement  a  prediction  market.  In  general,  the 
upfront  financial  burden  of  the  prediction  market  may  seem  quite  low,  making  it  an 
attractive  option  for  organizational  forecasting;  however,  organizations  must  understand 
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that  prediction  markets  require  a  great  deal  of  time  and  effort  for  successful  development 
and  operation.  Furthermore,  much  of  this  time  burden  will  likely  fall  on  existing 
personnel  (such  as  departmental  managers  or  higher)  to  ensure  selecting  the  most 
appropriate  and  useful  marketplace  questions  and  to  ensure  contracts  are  clear  and 
formed  in  such  a  way  as  to  draw  out  all  desired  information. 

Clearly,  cost  is  of  significant  importance  since  every  organization  must  abide  by 
some  form  of  budget  constraint.  Effectively,  there  is  finite  money  and  resources  available 
for  organizations  to  invest  in  prediction  markets.  As  with  all  sound  investments,  the 
benefits  of  the  investment  should  outweigh  the  cost  or  risk  of  the  investment.  Therefore, 
organizations  must  conduct  sufficient  analysis  and  give  great  thought  when  assessing  the 
feasibility  of  utilizing  a  prediction  market  for  organizational  gain. 

3.  Political  and  Legal  Considerations 

Political  or  legal  circumstances  may  affect  one’s  decision  to  use  prediction 
markets.  The  DARPA  prediction  market  effort  to  predict  and  prevent  terrorist  attacks  is 
one  common  example  used  to  highlight  an  unfavorable  political  condition  for  market 
utilization.  DARPA’s  FutureMap  program  came  under  Congressional  and  media  scrutiny 
on  the  following  bases:  1)  the  markets  were  unethical  and  2)  adversaries  could  potentially 
manipulate  markets  in  their  favor.  As  a  result,  DARPA  shut  down  the  FutureMap 
program  (Fooney,  2003). 

Fegal  obstacles  can  prove  equally  challenging  when  assessing  the  overall 
environment  for  prediction  market  utilization.  Gambling  and  trade  regulations  pose 
serious  challenges  to  adopting  prediction  markets  for  decision-making  purposes.  As  a 
result,  organizations  must  thoroughly  consider  the  market’s  trade  format  as  well  as  the 
means  of  incentivizing  participants.  According  to  Chen  (2005),  “legal  issues  of  gambling 
make  most  public  information  markets  in  the  United  States  only  play  money”  (p.  121)M 


This  does  not  imply  that  play  money  markets  are  not  as  effective  or  accurate  as  real  money  markets. 
Indeed,  a  number  of  experiments  have  proven  differences  in  accuracy  can  be  negligible  between  real  and 
play  money  markets.  A  subsequent  section  pertaining  to  participatory  incentives  develops  this  idea  more 
fully. 
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V.  DESIGN  AND  IMPLEMENTATION 


A.  DESIGN 

1.  Introduction/Overview 

According  to  Wolfers  and  Zitzewitz  (2004),  “The  success  of  predietion  markets 
.  .  .  depend[s]  on  their  design  and  implementation”  (p.  120).  Furthermore,  in  designing  a 
prediction  market,  an  organization  must  consider  numerous  factors  and  details.  The  next 
five  sections  discuss  critical  aspects  of  market  design,  to  include: 

•  Claim  Definition 

•  Claim  Structure 

•  Trading  Mechanisms 

•  Participation 

•  Real  money,  Play  money  and  Associated  Incentives 

2.  Claim  Definition 

Claim  definition  is  one  of  the  most  critical  and  challenging  aspects  of  prediction 
market  design  and  development.  Claim  definition  is  the  means  by  which  a  market 
designer  communicates  his  desired  purpose  or  claim  end  state  to  market  participants. 
Practically  applied,  the  claim  definition  is  the  question  or  statement  posed  to  the 
marketplace  traders.  Even  with  an  abundance  of  useful  knowledge  readily  available  for 
aggregation,  a  poorly  defined  claim  can  render  a  market  ineffective  by  nullifying 
potential  market  benefits  and  usefulness.  Wolfers  and  Zitzewitz  (2004)  succinctly  state, 
“[f]or  a  predietion  market  to  work  well,  eontracts  must  be  clear,  easily  understood  and 
easily  adjudicated”  (p.  120). 

To  be  effective  and  legitimate,  prediction  markets  must  account  for  all  possible 
event  outcomes.  In  practice,  this  means  that  if  one  is  interested  in  assessing  the 
probability  of  a  given  football  team  beating  another,  market  designers  must  also  account 
for  the  possibility  of  a  tie.  This  allows  traders  to  fully  interact  with  one  another  without 
doubt  or  uneertainty  in  the  eontraet’s  designed  end  state.  Aeeordingly,  eontracts  must 
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have  a  clear  and  definite  end  state  that  bears  no  element  of  doubt  or  uncertainty,  and  they 
should  have  an  associated  pre-determined  timeframe  for  which  they  are  active.  For 
example,  the  following  claim  “The  Cubs  will  win  a  championship”  does  not  provide  a 
clear  and  definite  end  state  because  it  does  not  specify  whether  it  refers  to  the  World 
Series  (the  major  league  baseball  championship),  the  National  League  Championship,  or 
something  else  altogether;  each  are  viable  options  that  any  given  individual  could 
perceive  as  satisfying  the  claim.  Therefore,  the  claim  in  question  is  unclear  and  open  to 
interpretation.  Additionally,  the  claim  “The  Cubs  will  win  the  World  Series”  does  not 
provide  a  pre-determined  timeframe  for  which  the  claim  is  active.  A  more  appropriate 
claim  is  “The  Cubs  will  win  the  World  Series  before  2012.”  This  claim  is  clearly  defined, 
and  has  a  definite  end  state  and  a  predetermined  timeframe.  Though  challenging, 
organizations  using  prediction  markets  must  clearly  define  all  claims  for  the  market  to 
remain  effective,  efficient,  and  manageable. 

3.  Claim  Structure 

Participants  make  claims  about  the  future  through  the  prediction  market 
securities.  These  claims  have  associated  prices  and  are  traded  in  the  prediction  market  in 
much  the  same  way  as  stocks  are  traded  in  public  stock  markets.  The  claim  structure  is 
the  means  of  assigning  the  associated  payoff  to  market  traders.  It  depends  on  the 
security’s  objective  as  determined  by  the  claim  definition.  The  most  common  claim 
structures  are  winner-take-all,  index,  and  spread.  The  following  paragraphs  describe  each 
of  these  three  structures. 

a.  Winner-take-all 

A  winner-take-all  structure  determines  the  likelihood  or  probability  of  a 
single  event’s  occurrence.  A  political  election  is  a  commonly  cited  example.  Winner- 
take-all  should  be  the  claim  structure  for  determining  the  probability  that  a  party  or 
candidate  will  win.  As  described  by  Wolfers  and  Zitzewitz  (2004),  “The  price  on  a 
winner-take-all  market  represents  the  market’s  expectation  of  the  probability  that  an 
event  will  occur  (assuming  risk  neutrality)”  (p.  109). 
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b. 


Index 


An  index  claim  is  useful  in  determining  the  expected  value  of  a  continuous 
random  variable  (Chen,  2005,  p.  9).  The  payoff  for  an  index  claim  is  variable  and 
depends  on  the  event’s  outcome  (e.g.,  a  payment  equal  to  the  percentage  of  the  popular 
vote  received  by  a  political  party  or  candidate).  Consistent  with  the  above  election 
example,  an  index  claim  has  the  ability  to  forecast  the  percentage  of  votes  for  a  given 
party  or  candidate  during  a  given  election.  Wolfers  and  Zitzewitz  (2004)  define  an 
index’s  market  price  as  “the  mean  value  that  the  market  assigns  to  the  outcome”  (p.  109). 

c.  Spread 

The  spread  claim  forecasts  the  probability  of  a  future  event’s  occurrence 
by  adjusting  the  margin  from  a  fixed  suggested  probability  for  a  given  claim’s 
occurrence.  To  remain  consistent  with  the  example  above,  a  spread  claim  is  useful  in 
determining  whether  a  particular  political  party  or  candidate  is  likely  to  receive  more  (or 
less)  than  a  given  percentage  of  votes  during  a  given  election.  Deferring  to  Wolfers  and 
Zitzewitz  (2004)  helps  provide  a  clearer  description  of  this  claim,  in  addition  to  the 
meaning  of  its  results: 

In  spread  betting,  the  price  of  the  bet  is  fixed,  but  the  size  of  the  spread 
can  adjust.  When  spread  betting  is  combined  with  an  even-money  bet  (that 
is,  winners  double  their  money  while  losers  receive  zero),  the  outcome  can 
yield  the  market’s  expectation  of  the  median  outcome,  because  this  is  only 
a  fair  bet  if  a  payoff  is  as  likely  to  occur  as  not.  (p.  109) 

Although  the  three  security  claim  types  discussed  are  mutually  exclusive, 
market  managers  may  elect  to  use  more  than  one  of  the  claim  types  to  evaluate 
information  associated  with  a  given  market.  For  the  example  used  throughout  this 
discussion,  each  claim  type  offers  a  different  perspective  of  the  aggregated  information 
(essentially,  with  respect  to  the  same  event).  This  can  be  quite  useful  for  assessing  and 
analyzing  market  results  for  organizations  to  make  better-informed  decisions. 
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4.  Trading  Mechanisms 

Appropriate  mechanism  design  is  essential  when  developing  a  successful  market. 
In  short,  the  market  trading  mechanism  is  what  determines  market  conduct  among  all 
participants;  it  is  that  which  connects  buyers  and  sellers  for  market  trading.  Interestingly, 
many  trading  mechanisms,  such  as  those  utilizing  an  automated  market  maker,  can 
function  without  requiring  multiple  traders  to  directly  interact  with  one  another.  Instead, 
such  mechanisms  utilize  a  computerized  programming  device  or  mathematical  algorithm 
that  acts  as  a  universal  buyer/seller  for  any  given  transaction.  The  advantage  of  the 
automated  market  maker  is  that  a  trader  can  buy  or  sell  at  any  time,  regardless  of  whether 
other  traders  want  to  buy  or  sell. 

By  far,  the  most  common  trading  mechanisms  are  the  continuous  double  auction 
(CD A)  and  market  scoring  rules  (MSR).  In  addition,  the  dynamic  pari-mutuel  system 
(DPM)  holds  credence  with  many  experts.  Each  of  these  mechanisms  is  a  viable 
candidate  for  market  implementation;  however,  organizations  must  understand  the 
benefits  and  limitations  of  each  to  select  the  most  suitable  mechanism  for  a  given  market. 
Descriptions  of  each  mechanism  follow. 

a.  CDA 

CDA  is  a  widespread  prediction  market  trading  mechanism.  It  functions 
much  like  common  financial  stock  exchanges,  in  which  buyers  make  offers  or  bids  on  a 
security  via  buy  orders,  and  sellers  post  an  asking  price  via  sell  orders.  When  the  buyer 
and  seller  prices  meet,  there  is  agreement  for  the  sale  or  trade  of  a  given  security.  The  key 
benefit  to  CDA  is  its  familiarity  for  market  managers  and  participants.  Additionally, 
CDA  is  appropriate  when  organizations  aspire  to  forecast  a  specific  point  within  a  broad 
range  of  numbers.  However,  the  critical  drawback  of  CDA  is  its  inability  to  maintain 
liquidity  in  thin  markets. 

Some  CDA  markets  use  a  market  maker  (CDAwMM)  to  help  maintain 
liquidity,  but  they  possess  a  unique  and  severe  drawback  in  the  form  of  added  risk  for  the 
auctioneer  or  market  owner  (Chen,  2005,  p.  127).  As  Chen  points  out,  “The  auctioneer 
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may  lose  considerable  amounts  of  money  depends  [5/c]  on  what  happens  in  the  future. 
The  cost  of  the  auctioneer  is  not  bounded”  (p.  127). 

Though  slightly  more  complex  than  traditional  CDA’s,  NewsFutures  is 
perhaps  the  most  well  known  example  of  a  CDA  marketplace.  NewsFutures  utilizes  an 
automated  market  maker  to  improve  liquidity  within  its  markets,  but  traders  buy  and  sell 
shares  directly  from  one  another  through  an  automatic  trade-offer  queuing  process. 

b.  MSR 

To  overcome  liquidity  shortcomings  associated  with  traditional  CDAs, 
Robin  Hanson  proposed  a  scoring  rules  mechanism,  commonly  referred  to  as  market 
scoring  rules  (p.  107).  Wolfers  and  Zitzewitz  (2004)  explain  that  Hanson  suggests  using  a 
set  of  scoring  rules  to  allow  market  participants  to  trade  on  “simultaneous  predictions 
over  many  combinations  of  outcomes”  (p.  120).  This  allows  for  combining  market  events 
for  a  more  realistic  forecasting  assessment  rather  than  simplistically  isolating  individual 
events.  According  to  Wolfers  and  Zitzewitz,  “instead  of  requiring  separate  markets  for 
each  combination  of  possible  outcomes,  traders  effectively  bet  that  the  sum  of  their  errors 
over  all  predictions  will  be  lower”  (p.  120).  Hanson’s  MSR  effectively  acts  as  an 
automated  market  maker  that  rewards  participants  for  improving  the  overall  quality  of 
market  forecasts  in  a  stepwise  manner.  Chen  (2005)  provides  the  following  description  of 
Hanson’s  MSR: 

MSR  maintains  a  probability  distribution  across  all  events.  Anyone  who 
believes  that  the  probability  distribution  is  wrong  can  change  it  at  any 
time.  The  person  then  receives  a  payment  [.  .  .]  according  to  a  scoring  rule, 
and  in  return,  agrees  to  pay  the  next  person  who  changes  the  distribution. 

(p.  127) 

MSR  is  useful  in  forecasting  a  broader  or  more  complex  set  of  outcomes. 
Additionally,  MSR  guarantees  liquidity  since  no  buyer- seller  pricing  matching  is  required 
for  trading  to  occur.  However,  because  of  its  design,  MSR  is  not  appropriate  when 
forecasting  figures  within  a  continuous  range  of  numbers.  ^3  Although  there  is  some  risk 

^^3  This  often  is  overcome,  to  some  extent,  by  offering  multiple  contract  options,  each  associated  with  a 
given  smaller  range  within  the  entire  range  of  consideration.  Collectively,  the  smaller  ranges  are 
continuous,  and  they  encompass  the  entire  desired  range  of  consideration. 
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or  cost  required  for  subsidizing  the  initial  market  bet(s),  MSR  limits  the  market 
manager’s  maximum  cost  to  the  initial  subsidy.  Participants  assume  all  other  trading  risk 
by  paying  (receiving)  funds  to  (from)  one  another  via  the  market  maker  and  according  to 
the  market’s  scoring  rules. 

Inkling’s  public  marketplace  is  a  good  example  of  MSR.  The  MSR  acts  as 
an  automated  market  maker  by  which  users  can  conduct  trades  via  the  simple  web- 
interface.  Participants  can  always  buy  or  sell  shares  within  a  given  market,  assuming  they 
have  sufficient  wealth  in  their  portfolios.  The  Inkling  platform  is  discussed  further  in 
Chapter  VI  of  this  thesis, 

c.  DPM 

Seeking  to  overcome  potential  liquidity  limitations  associated  with 
traditional  market  mechanisms,  David  Pennock  proposed  the  dynamic  pari-mutuel 
system,  and  its  associated  differential  equations.  In  effect,  DPM  acts  as  an  automated 
market  maker  since  DPM  operates  based  on  a  price  function  that  reflects  a  continuously 
updated  probability  for  a  given  event’s  occurrence.  Share  prices  update  dynamically 
according  to  the  price  function,  which  can  be  fixed  in  one  of  two  primary  ways:  money- 
ratio  or  share -ratio.  The  money-ratio  price  function  “defines  the  ratio  of  any  two  stock 
prices  in  the  same  market  as  always  equal  to  the  ratio  of  money  invested  in  the  stocks” 
(Chen,  Pennock,  &  Kasturi,  2008,  p.  4).  With  the  share -ratio,  the  price  function  sets  the 
share  prices  “to  equate  the  ratio  of  prices  of  any  two  securities  by  the  ratio  of  number  of 
shares  outstanding  for  the  two  securities  at  any  time  of  the  market”  (Chen,  Pennock,  & 
Kasturi,  2008,  p.  2).  Because  of  the  ways  in  which  the  price  functions  work,  market 
probabilities  are  not  explicitly  inferable  from  given  market  prices.  Rather,  other 
differential  equations  translate  market  prices  into  given  event  probabilities,  if  so  desired. 


The  practical  experiment  associated  with  this  thesis  relied  upon  Inkling’s  platform. 
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Whether  real  or  play,  money  from  the  sale  of  market  securities  moves  to  a 
collective  market  pot.  For  market  payout,  the  market  manager  deducts  any  maintenance 
and  transaction  fees  from  the  pot  and  then  disperses  funds  to  all  market  winners,  with 
market  losers  receiving  nothing.  Chen  (2005)  offers  a  good  description  of  the  DPM 
payout  process: 

After  the  true  outcome  is  revealed,  all  the  money  that  is  lost  by  those  who 
bet  on  the  incorrect  outcome  is  redistributed  to  those  who  bet  on  the 
correct  outcome.  [...]  Unlike  a  pari-mutual  market,  where  each  dollar 
always  buys  an  equal  share  of  the  payoff,  each  dollar  that  people  wager  in 
a  DPM  buys  a  variable  share  of  the  payoff  depending  on  the  state  of  the 
market,  (pp.  127-128) 

Familiarity  and  simplicity  are  benefits  of  the  DPM  mechanism.  In 
addition,  unlike  traditional  pari-mutuel  systems  that  incentivize  late  trading,  DPM 
incentivizes  participants  to  reveal  good  information  early.  This  occurs  because  the 
amount  invested  in  the  market  can  only  increase  over  time,  which  means  that  market 
probability  predictions  are  more  sensitive  to  a  given  investment  earlier  than  later. 
Additionally,  the  DPM  mechanism  has  infinite  liquidity,  as  there  is  no  requirement  for 
buyer/seller  order  matching  (Chen,  2005,  p.  127).  Furthermore,  DPM  generally  is 
desirable  when  organizations  seek  to  forecast  a  specific  point  within  a  broad  range  of 
numbers.  Finally,  similar  to  MSR  there  is  no  manager  risk  or  cost  beyond  the  required 
subsidy  for  the  initial  trade(s). 

It  is  unclear  whether  any  current  public  marketplaces  exclusively  utilize 
DPM.  However,  the  Yahoo! -O’ Reilly  Tech  Buzz  Game  is  one  of  the  most  notable  DPM 
applications.  The  Tech  Buzz  Game  operated  from  2005  to  2008  with  dual  purposes:  “One 
[was]  to  evaluate  the  power  of  prediction  markets  to  forecast  high-tech  trends.  [.  .  .]  The 
other  [.  .  .]  [was]  to  field  test  the  dynamic  pari-mutuel  market”  (Mangold,  et  ah,  2005,  p. 
94).  Traders  conducted  on-demand  transactions  by  buying  and  selling  shares  from  one 
another  and/or  from  a  market  maker — the  marketplace  interface  smoothly  masked  the 
market  mechanism’s  mechanics.  Traders  simply  placed  orders  and  watched  transactions 
occur  instantly,  without  knowledge  of  whether  a  given  trade  occurred  with  another  trader 
or  with  the  market  maker. 
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Selecting  an  appropriate  market  mechanism  is  central  to  the  design 
process.  Organizations  must  fully  understand  their  own  objectives,  intended  scope,  and 
other  likely  market  limitations  when  selecting  the  market  mechanism  for  a  given 
marketplace.  Additionally,  there  is  no  such  thing  as  a  “one-size-fits-all”  design 
appropriate  for  all  organizations.  For  a  given  organization,  determining  which  trading 
mechanism  is  most  appropriate  depends  on  two  primary  considerations: 

•  How  large  is  the  potential  participant  pool? 

•  What  type  of  information  is  the  organization  attempting  to  forecast? 

If  the  potential  participant  pool  is  small,  the  resultant  design  implication  is 
that  an  automated  market  maker  is  necessary  to  promote  liquidity  and  avoid  high  person- 
to-person  bid/ask  spreads.  Additionally,  the  organization’s  selection  of  market 
mechanism  should  depend  on  the  type  of  information  the  organization  seeks  to  forecast. 
If  an  organization  is  forecasting  mutually  exclusive  or  discrete  outcomes,  MSR  is  an 
appropriate  mechanism.  However,  MSR  does  not  allow  organizations  to  forecast 
continuous  organizational  metrics.  Rather,  the  CD  A  or  DPM  mechanisms  are  more 
appropriate  when  organizations  want  to  pinpoint  a  continuous  number. 

Borrowed  from  Chen  (2005),  Table  2  summarizes  the  above  mechanisms 
and  their  relative  liquidity  and  risk. 


44 


Table  2.  Comparison  of  Trading  Mechanisms  (From  Chen,  2005,  p.  129) 


Trading  Mechanism 

Liquidity 

Market  Manager  Risk/Cost 

CDA 

Illiquidity  when  market  is  thin 

No  risk,  only  matching  orders. 

CDAwMM 

Guaranteed  liquidity 

Market  owner  has  risk,  can  incur 

unbounded  cost. 

MSR 

Guaranteed  liquidity 

Market  owner  has  limited  risk,  can 

incur  bounded  cost. 

Market  owner  needs  a  predetermined 

DPM 

Guaranteed  liquidity 

cost  to  start  the  market.  No  risk  after 

the  market  is  started. 

5.  Participation 

In  determining  which  individuals  should  participate  in  a  given  prediction  market, 
organizations  should  look  beyond  an  expert-only  participant  pool,  and  elicit  participation 
from  any  organizational  member  with  access  to  pertinent  market  information.  Intuitively, 
wider  participation  pools  generally  provide  a  greater  knowledge  base  than  do  narrow 
participation  pools.  Moreover,  prediction  markets  can  thrive  only  when  liquidity  exists 
from  a  diverse  pool  of  active  participants.  Put  simply,  markets  need  uninformed 
participants — those  participants  whose  topical  knowledge  base  provides  no  further 
insight  into  a  given  market  security’s  potential  outcome — in  addition  to  informed 
participants  to  ensure  market  liquidity  and  ongoing  activity.  This  diversity  creates 
potential  for  profit  and,  thus,  creates  incentive  for  all  participants  to  engage  actively.  In 
seeking  only  experts  to  participate,  organizations  may  miss  this  necessary  diversity  if 
expert  opinion  is  relatively  uniform,  and  participants  face  the  motivation  challenges 
discussed  shortly. 

Eliciting  participation  is  another  crucial  element  when  operating  a  prediction 
market.  Often,  the  primary  motivation  factors  driving  market  participation  are 
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entertainment  and  personal  confidence  or  pride  in  one’s  expertise.  People  generally 
choose  to  participate  in  markets  and  auctions  when  there  are  prizes  or  when  the  market  is 
attractive  and  entertaining.  Common  examples  are  sports  and  movie  wagering  markets, 
which  are  only  a  step  away  from  gambling  in  terms  of  individual  motivation  for 
participation.  Wolfers  and  Zitzewitz  (2004)  appear  to  support  this  notion  based  on  the 
following  quote,  “the  ‘play  money’  exchanges  and  sports  gambling  industry  both  suggest 
that  it  may  be  possible  to  motivate  [.  .  .]  trading  simply  through  the  thrill  of  pitting  one’s 
judgment  against  others,  and  being  able  to  win  a  monetary  prize  may  sharpen  this 
motivation”  (p.  121). 

Personal  pride  or  confidence  in  one’s  expertise  can  also  lead  individuals  to 
participate  in  prediction  markets  based  on  a  presumed  superiority  in  obtaining  or 
processing  available  knowledge.  If  all  participants  believed  they  were  no  better  than 
anyone  else  at  obtaining  or  processing  available  information,  then  there  is  no  motivation 
to  conduct  trades  or  seek  new  information.  Moreover,  the  nature  of  available  information 
plays  a  significant  role  among  participants  in  perceiving  their  own  relative  understanding 
of  that  information.  For  instance,  people  are  hesitant  to  conduct  trades  when  they  believe 
there  is  a  subgroup  of  participants  possessing  insider  information.  Skepticism  can  affect 
the  group  as  a  whole,  preventing  would-be  participants  from  conducting  trades.  In  this 
environment,  the  market  can  break  down  and  become  completely  ineffective.  The 
following  passage  from  Wolfers  and  Zitzewitz  (2004)  underscores  the  importance  of 
these  revelations: 

These  insights  suggest  that  some  prediction  markets  will  work  better  when 
they  concern  events  that  are  widely  discussed,  since  trading  on  such  events 
will  have  higher  entertainment  value  and  there  will  be  more  information 
on  whose  interpretation  traders  can  disagree.  Ambiguous  public 
information  may  be  better  in  motivating  trade  than  private  information, 
especially  if  the  private  information  is  concentrated,  since  a  cadre  of 
highly  informed  traders  can  easily  drive  out  the  partly  informed, 
repressing  trade  to  the  point  that  the  market  barely  exists,  (p.  121) 

6.  Real  Money,  Play  Money  and  Associated  Incentives 

As  discussed  in  the  previous  section,  participation  is  vital  for  a  prediction 
market’s  success.  However,  various  motivating  forces  drive  participants  to  engage 
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actively  in  a  given  market.  According  to  Schrieber  (2004),  “Although  many  informed 
individuals  are  motivated  to  trade  simply  out  of  an  intrinsic  enjoyment  [.  .  .]  remuneration 
may  be  required  to  extract  timely  and  accurate  information  from  others”  (p.  37).  When 
considering  participation,  one  likely  thinks  of  potential  participatory  incentives.  Such 
incentives  can  be  useful  for  enticing  participants  who  otherwise  would  not  engage  in  the 
prediction  market.  To  this  point,  successful  markets  have  utilized  cash  and  non-cash 
incentives  to  elicit  participation.  Moreover,  these  incentives  are  generally  linked  to  a 
market’s  assets.  Market  assets  are  defined  by  two  specific  market  structures:  real  money 
and  play  money.  The  following  paragraphs  provide  real  and  play  money  market 
descriptions,  along  with  discussions  on  cash  and  non-cash  incentives. 

Cash  incentives  are  most  commonly  associated  with  real  money  markets.  Real 
money  markets  require  participants  to  use  their  own  money,  though  perhaps  subsidized 
by  the  organization,  to  conduct  trades.  This  structure  may  provide  strong  incentives  for 
some  individuals,  as  it  can  appeal  to  their  sense  of  competition  and  desire  for  profit. 
Additionally,  this  strong  incentive  for  participants  to  perform  well  also  implies  the 
market  itself  fares  better  with  frequent  and  improved  forecasting  performances.  The 
previous  section  regarding  legal  and  political  constraints  has  already  developed  the  major 
challenge  this  reward  structure  presents.  Real  money  markets  are  not  acceptable  for  all 
markets,  environments,  or  organizations  because  of  their  similarity  to  gambling,  concerns 
over  market  manipulation,  and  ethical  concerns  over  requiring  employees  to  provide 
personal  capital  for  organizational  benefit. 

Play  money  markets  are  among  the  most  prevalent  for  public  and  professional 
use.  These  markets  usually  provide  some  initial  allotment  of  play  money  (Chen,  2005,  p. 
130),  which  participants  are  free  to  use  within  the  market  as  they  see  fit.  As  discussed  by 
Wolfers  and  Zitzewitz  (2004),  an  added  benefit  of  play  money  markets  is  the  flexibility 
they  offer:  “Play  money  contracts  [.  .  .]  offer  more  freedom  to  experiment  with  different 
kinds  of  contracts”  without  participants  bearing  a  monetary  risk  or  penalty  (p.  121). 
Generally,  play  money  markets  use  other  incentives,  such  as  prizes  or  points,  to  elicit 
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participation.  Chen  (2005)  cites  one  good  example,  “participants  of  Newsfutures’ 
prediction  markets  can  use  the  play  money  they  earned  to  buy  some  items  in  an  online 
auction  shop”  (p.  130). 

Although  non-cash  incentives  can  be  associated  with  real  money  markets,  they 
predominate  in  play  money  markets  as  the  primary  motivating  force(s).  The  most 
common  non-cash  incentives  are  prizes  or  points — for  use  as  currency  in  a  marketplace 
store — that  reward  the  historically  most  accurate  traders  or  most  active  traders  for  their 
participation.  Additionally,  intrinsic  motivating  incentives  that  drive  marketplace  traders 
to  continue  participating  over  time  include  personal  pride,  natural  competition,  and  a 
means  of  anonymously  voicing  opinions.  These  motivators  generally  factor  into  most 
markets,  regardless  of  whether  they  are  real  or  play  money. 

When  establishing  a  real  or  play  money  market,  initial  endowments  may  be  useful 
because  they  provide  similar  feel  and  motivation  to  those  of  real  money  markets,  but  the 
organization  absolves  risk  from  individual  participants  and  removes  ethical  concerns  over 
participants  using  their  personal  money  for  organizational  benefit.  Generally,  this 
structure  allows  participants  to  profit,  in  some  form,  from  amounts  raised  beyond  the 
initial  endowment.  Although  this  particular  reward  structure  may  address  some  ethical 
concerns,  Chen  (2005)  warns  that  others  still  exist:  “For  public  information  markets,  legal 
and  political  concerns  often  prevent  markets  from  using  real  money”  (p.  130). 

In  evaluating  the  reward  structures  above,  prediction  market  experts  have  posed  a 
common  question  regarding  accuracy:  Are  play  money  markets  as  accurate  as  real  money 
markets?  Although  insufficient  empirical  evidence  exists  for  irrefutable  confirmation, 
various  studies  do  support  the  claim  that  play  money  markets  are  nearly  equally  as 
accurate  as  real  money  markets  because  play  money  “wealth”  can  only  be  accumulated 
through  a  history  of  accurate  predictions  (Wolfers  &  Zitzewitz,  2004,  p.  121).  In  support 
of  this  claim,  Wolfers  and  Zitzewitz  offer  the  following: 

In  a  suggestive  experiment,  Servan-Schreiber,  Wolfers,  Pennock  and 
Galebach  compared  the  predictive  power  of  the  prices  from  real  money 
and  play  money  exchanges  over  the  2003  NFL  football  season,  finding 
that  both  yielded  predictions  that  were  approximately  equally  accurate,  (p. 

121) 
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Clearly,  several  options  exist  for  market  designers  to  elicit  participant 
involvement.  The  choice  of  which  structure  to  use  will  not  always  be  clear.  Additionally, 
external  forces  or  pressures  may  drive  this  choice,  such  as  legal  or  political  issues.  In  any 
event,  the  choice  may  prove  to  be  one  of  preference  rather  than  essence.  Tziralis  and 
Tatsiopoulos  (2007)  state,  “The  decision  of  whether  to  use  monetary  ...  or  non-monetary 
rewards  ...  is  up  to  designer  and  remains  more  an  art  than  a  science”  (p.  256). 

7.  Review 

The  previous  sections  covered  several  aspects  of  prediction  market  design, 
including:  1)  claim  definition;  2)  claim  structure;  3)  trading  mechanisms;  4)  participation; 
and  5)  real  money,  play  money  and  associated  incentives.  Table  3  provides  a  summary  of 
the  various  design  aspects  discussed  above. 
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Table  3.  Summary  of  Prediction  Market  Design  Aspects  and  Considerations 


Design  Aspect 

Considerations 

Claim  Definition 

Clear  and  definite  end  state 

Pre-determined  timeframe 

Winner-take-all 

Claim  Structure 

Index 

Spread 

CDA 

Trading  Mechanisms 

DPM 

MSR 

Participation 

Diverse  body  of  participants 

Motivating  forces 

Real  money 

Market  Format 

Play  money 

Initial  Endowment 

Cash  incentives 

Non-cash  incentives 

Incentives 

Prizes 

Competition 

Pride 

Ability  to  voice  opinions  anonymously 

B.  IMPLEMENTATION 


To  introduce  market  implementation,  Chen  (2005)  offers  the  following,  “Market 
implementation  is  the  construction  of  the  new  [prediction]  market  and  the  delivery  of  the 
market  into  operation”  (p.  134).  In  planning  to  implement  a  prediction  market,  an 
organization  should  consider  three  primary  phases  of  implementation:  1)  preparation,  2) 
conduct,  and  3)  support.  The  three  phases  of  implementation  offer  a  useful  structure  for 
assessing  potential  prediction  market  limitations,  challenges  and  risks.  Organizations 
should  assess  each  phase  of  the  implementation  process.  The  following  subsections 
highlight  issues  of  concern  for  each  respective  phase. 
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1.  Preparation 

The  preparation  phase  encompasses  several  aspects  of  prediction  market 
implementation,  from  feasibility  assessment,  to  budgetary  considerations,  to  establishing 
a  pilot  market.  Preparing  to  implement  a  prediction  market  is  a  challenging  and 
meticulous  process.  Appropriate  support  systems,  such  as  networks  and  software 
packages,  must  be  in  place.  An  organization  must  determine  its  target  audience,  what 
market  framework  to  use,  and  what  types  of  questions  will  best  meet  market  objectives. 
Additionally,  an  organization  must  have  a  good  sense  of  the  type(s)  of  incentives  that  are 
best  suited  for  the  target  audience  to  meet  market  objectives.  Cost  assessments  should 
provide  useful  information  for  determining  how  to  fund  the  market,  as  well  as  for 
determining  market  scope  limitations.  Finally,  before  introducing  a  market  to  the  target 
audience,  organizations  should  conduct  at  least  one  pilot  market  to  assess  adequacy  and 
monitor  for  undesirable  market  issues  or  attributes.  This  action  will  help  prevent 
participants  from  forming  an  unfavorable  perception  of  prediction  markets  because  of 
poor  market  design  or  improper  market  implementation. 

2.  Conduct 

One  potential  pitfall  when  opening  a  market  is  setting  initial  contract 
prices.  Contract  prices  should  be  set  at  a  reasonable  level  to  ensure  there  is  not  an 
overwhelming  arbitrage  opportunity  when  the  market  opens.  Moreover,  prices  generally 
are  a  direct  indicator  of  the  marketplace’s  assessed  probability  of  a  given  event’s 
occurrence.  Therefore,  improper  initial  pricing  can  give  early  traders  an  unfair  advantage 
to  trade  on  a  contract  that  is  widely  believed  to  be  under  or  overpriced.  Furthermore, 
initial  prices  can  affect  traders  perceptions  of  a  given  event’s  expected  probability.  Thus, 
poorly  set  initial  prices  can  adversely  affect  the  overall  market’s  initial  assessment  of  a 
given  event’s  probability  of  occurrence.  In  theory,  an  efficient  market  will  eventually 
overcome  such  problems. 

Next,  when  conducting  a  market,  an  organization  must  continually  monitor 
participation  rates  to  assess  the  effectiveness  of  incentives.  As  previously  discussed, 
incentives  can  significantly  affect  overall  participation.  Another  factor  that  can  affect 
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participation  is  selecting  relevant  and  clear  questions.  If  questions  are  unclear  or 
irrelevant,  participants  are  generally  less  inclined  to  trade. 

An  additional  issue  is  that  organizations  must  find  the  right  balance  of  questions 
that  will  adequately  aggregate  information  without  thinning  out  the  markets.  Due  to 
various  participatory  constraints,  such  as  limited  available  trading  time  or  personal 
interest  in  the  marketplace,  participants  may  conduct  some  fixed  number  of  total  trades 
within  the  overall  marketplace.  As  the  number  of  questions  increases,  the  average 
participation  rate  per  question  decreases.  This  leads  to  adverse  market  conditions  with 
improper  equilibrium  prices  due  to  the  market’s  thinning.  Therefore,  the  market  may  not 
efficiently  aggregate  all  available  information. 

A  final  element  to  consider  for  market  conduct  is  timing.  The  timing  of  when  and 
how  frequently  questions  are  released  may  affect  participation  rates,  in  part  reflecting 
geographic  divides  among  participants.  Furthermore,  poor  or  untimely  questions  can 
present  arbitrage  opportunities  for  those  individuals  who  are  first  to  trade  on  a  given 
contract. 


3.  Support 

As  with  any  process  or  service,  organizations  must  provide  ongoing  support  to 
ensure  the  market  operates  as  designed  and  desired.  The  support  phase  entails  all  details 
and  considerations  to  ensure  participants  remain  involved  and  that  the  market  operates 
optimally  to  aggregate  useful  information.  Chen  (2005)  highlights  potential  problems  and 
issues  arising  in  this  stage,  to  include  “dispute  [s]  over  market  trading  rules  and  traded 
contracts,  database  or  network  problems,  and  system  security  issues”  (pp.  134-135). 
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VI.  PRACTICAL  EXPERIMENT 


A.  EXPERIMENT  DESCRIPTION 

The  experiment  involved  running  a  pilot  market  to  begin  examining  whether  the 
Navy  should  consider  applying  prediction  markets  to  manpower  outcomes,  such  as 
recruiting,  retention,  and  re-enlistment  bonuses.  The  experiment  was  more  hastily 
designed  than  we  would  have  preferred.  We  faced  a  time  constraint  in  designing  and 
conducting  a  pilot  market  to  meet  our  Navy  sponsor’s  desires  and  our  graduation 
timeline.  Thus,  our  goals  were  to  perform  due  diligence  given  our  time  constraints  and  to 
document  our  lessons  learned. 

The  N1  directorate  identified  53  potential  participants  for  the  pilot,  spread  across 
both  operational  N1  personnel  and  the  research  organizations  supporting  Nl.  Because 
many  of  the  Nl  members  perform  similar  and  interrelated  work,  our  hope  was  that  a 
natural  competitive  spirit  would  create  a  cycle  of  market  activity,  engagement,  and 
participation.  The  Naval  Postgraduate  School  (NPS)  added  another  five  participants 
consisting  of  two  students  and  three  thesis  advisors.  Each  participant  received  an  initial 
endowment  of  $5,000  in  play  money.  Table  4  shows  the  distribution  of  potential 
participants  by  their  location,  office  code,  and  functional  responsibilities. 


Table  4.  Potential  Participant  Distribution  by  Location  and  Office  Code 


Location 

Office  Code 

Office  Functionai  Responsibiiity 

Washington,  DC 

N104 

Modeiing  and  Anaiysis  Branch 

18 

Washington,  DC 

NIZ 

Strategic  Affairs  Office 

3 

Washington,  DC 

N13 

Manpower,  Personnei,  Training  and  Education  Division 

2 

Washington,  DC 

N130 

Military  Pay  and  Compensation  Policy  Branch 

2 

Washington,  DC 

N133 

Nuclear  Propulsion  Program  Policy  Branch 

1 

Washington,  DC 

OCNR 

Research 

1 

Washington,  DC 

CNA 

Contractor 

7 

Washington,  DC 

LMi 

Contractor 

2 

Washington,  DC 

SAG 

Contractor 

2 

Washington,  DC 

Lewin 

Contractor 

1 

Washington,  DC 

SERCO 

Contractor 

1 

Miiiington,  TN 

NPRST 

Quick  Polls 

7 

Miiiington,  TN 

PMO 

Management 

2 

Miiiington,  TN 

BUPERS3 

Community  Manager/Career  Development 

4 

Monterey,  CA 

NPS 

Research 

5 

|Total 

58 
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As  described  in  Chapter  V,  user  anonymity  may  be  important  in  markets 
containing  potentially  controversial  questions  or  in  markets  whose  members  are  from 
varying  hierarchical  levels.  Participants  initially  received  a  generic  username  consisting 
of  their  first  initial  followed  by  their  last  name.  Upon  creating  their  accounts,  participants 
had  the  option  to  keep  their  generic  username  or  modify  it  for  fun  or  anonymity. 

One  unfortunate  circumstance  of  beginning  the  pilot  market  quickly  is  that  it  left 
little  time  to  discuss  incentives.  NPS  suggested  our  N1  sponsors  incentivize  participation 
with  non-monetary  rewards  to  the  top-gainer,  such  as  a  preferred  parking  space  or  lunch 
with  a  VIP.  However,  N1  chose  not  to  offer  non-monetary  incentives. 

B.  DESIGN 

The  limited  number  of  participants  was  the  primary  factor  in  deciding  to  adopt  a 
platform  using  an  automated  market  maker.  This  choice  allows  participants  to  trade 
without  using  the  bid/ask  process  required  in  the  CDA  mechanism.  Thus,  it  promotes 
market  liquidity.  The  choice  to  use  an  automated  market  maker  drove  the  subsequent 
choices  of  securities  types  and  their  associated  payoffs.  The  CEO  of  Inkling  Markets, 
Adam  Siegel,  generously  provided  his  time  to  answer  questions,  and  he  offered  the 
Inkling  Market  platform,  pro  bono,  for  the  pilot  prediction  market.  The  Inkling  Market 
platform  uses  Robin  Hanson’s  (2003)  market  scoring  rules  as  the  basis  for  their 
automated  market  maker. 

As  Berg  and  Proebsting  (2009)  point  out,  Hanson’s  market  scoring  rules  operate 
under  the  assumption  that  a  security  has  mutually  exclusive  outcomes.  Thus,  the 
automated  market  maker  is  useful  for  determining  binary  (yes/no),  interval,  or  discrete 
option  outcomes.  However,  the  algorithm  cannot  predict  the  market  equilibrium  of  a 
continuous  number,  which  is  one  function  Navy  N1  would  like  to  have  in  their  prediction 
markets.  We  were  able  to  circumvent  this  limitation,  to  some  extent,  by  allowing  traders 
to  choose  between  relatively  narrow  numerical  intervals. 

Hanson’s  market  scoring  rules  calls  for  subjectively  choosing  what  value  to 
assign  an  elasticity  constant,  “b”.  Berg  and  Proebsting  (2009)  describe  the  issue  in  this 
way: 
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The  elasticity  constant  b  controls  how  much  prices  change  for  a  given 
transaction  size  (measured  in  shares  or  cost).  Setting  b  is  a  vexing 
problem:  set  too  low,  the  market  prices  will  swing  wildly  on  any  trade, 
and  set  too  high,  the  market  may  not  move  enough  [to]  reasonably  reflect 
aggregate  opinions,  (p.  51) 

Inkling  Markets  chooses  to  set  b  to  .10  per  share  traded,  which  means  that,  the 
price  adjusts  up  or  down  by  ten  cents  per  share  purchased  or  sold,  respectively.  Adam 
Siegel  admitted  this  was  an  arbitrary  number,  but  he  indicated  that  it  has  worked  for  their 
markets. 

The  security  payoffs  were  designed  as  winner-take-all.  Thus,  the  shares 
representing  the  correct  outcome  were  worth  $100  while  the  shares  representing  incorrect 
outcome(s)  were  worth  $0.  The  selection  of  winner-take-all  payoffs  was  required  for 
Inkling’s  automated  market  maker  using  Hanson’s  market  scoring  rules,  where  a  market 
involves  a  set  of  mutually  exclusive  outcomes.  Because  the  algorithm  assumes  only 
mutually  exclusive  outcomes  are  possible,  it  precludes  using  an  index  or  spread  payoff. 

C.  MARKET  MANAGEMENT 

We  tailored  and  sent  out  an  Inkling  Inc.  memo  to  demonstrate  executive 
sponsorship  to  participants.  Additionally,  we  tailored  an  Inkling  Inc.  introductory 
PowerPoint  slide  show  to  serve  as  an  introduction  to  prediction  markets  and  the  Inkling 
Markets  platform.  We  emphasized  the  explanation  of  short-sales  in  this  tutorial  to  ensure 
participants  were  aware  of  what  they  are  and  how  to  execute  them. 

1.  Claim  Selection 

The  overall  intent  was  to  ask  questions  to  assess  what  was  actually  taking  place 
with  respect  to  Navy  active  duty  entry  and  retention.  Generally,  the  United  States 
economy  was  in  relatively  poor  condition;  higher  than  normal  unemployment  rates  were 
coupled  with  declining  stock  market  prices  and  housing  values.  These  non-normal 
conditions  made  it  difficult  for  the  Navy  to  forecast  retention  rates  using  historical  data 
because  the  economic  conditions  were  much  different  from  the  past. 
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Taken  together,  this  seemed  to  be  an  ideal  time  to  explore  the  capability  for 
prediction  markets  to  forecast  retention.  However,  it  proved  difficult  for  Navy  N1  to 
identify  four  appropriate  securities  for  the  pilot.  The  N1  pilot  faced  two  inherently 
difficult  issues.  The  first  was  timing.  The  pilot  was  scheduled  to  last  just  under  two 
months  and  conclude  at  the  end  of  the  federal  government  fiscal  year  on  September  30, 
2009.  This  did  not  seem  to  be  problematic  at  the  outset,  yet  it  turned  out  to  be  so.  N1 
leadership  had  already  implemented  policy  changes  to  achieve  end  of  fiscal  year  goals. 
For  example,  N1  initially  wanted  to  determine  Nuke  Zone  A  retention  rates.  Upon  further 
investigation,  the  Navy  had  just  suspended  re-enlistment  bonuses  for  the  remainder  of  the 
fiscal  year  due  to  high  retention  rates  and  budget  constraints.  Therefore,  the  Nuke  Zone  A 
retention  rate  security  was  uninteresting:  rational  candidates  would  wait  until  the 
following  fiscal  year  to  re-enlist  when  they  would  be  eligible  to  receive  a  substantial 
bonus  for  re-enlisting,  thus  any  change  in  the  retention  rate  from  the  current  level  would 
only  reflect  those  leaving  the  Navy.  The  N1  and  NFS  consensus  was  to  discard  this 
candidate  question. 

The  other  issue  was  bureaucratic  in  nature.  One  candidate  question  involved  the 
number  of  people  waiting  to  join  the  Navy  via  the  Delayed  Entry  Program  (DEP).  While 
attempting  to  determine  the  inputs  to  this  calculation,  we  learned  there  were  at  least  two 
different  DEP  calculations  to  meet  differing  organizational  purposes.  As  a  result,  there 
would  be  confusion  over  the  security’s  definition. 

Ultimately,  the  N1  pilot  involved  two  direct  measures  for  Navy  entry  and 
retention.  These  were  markets  regarding  Navy  endstrength  and  the  following  year’s 
enlisted  accession  goal.  The  pilot  also  consisted  of  two  indirect  economic  measures  that 
may  affect  sailors’  decision  to  stay  in  or  leave  the  Navy.  The  two  indirect  measures  were 
for  the  national  unemployment  rate  and  the  Dow  Jones  Industrial  Average  (DJIA).  We 
also  employed  short-term  markets,  involving  primarily  sports  and  entertainment 
questions,  in  hopes  of  keeping  participants  interested  and  active  in  the  marketplace.  Table 
5  shows  the  questions  asked,  dates  the  markets  began  and  ended,  contract  type,  contract 
choices  and  initial  market  prices. 
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Table  5.  N1  Prediction  Market  Pilot  Questions 


Question 

Date  Began 

Date  Ended 

Contract  Type 

Contracts 

Initial  Market  Price 

What  will  be  the  Navy's  endstrength 
(for  officers  and  enlisted  personnel) 
for  FY2009?  (See  details  in  Market 
Information) 

Aug.  10,  2009 

Sep.  30,  2009 

Multiple-choice 

Less  than  330,200 

16.67% 

330,200-330,350 

16.67% 

330,350-330,500 

16.67% 

330,500-330,650 

16.67% 

330,650-330,800 

16.67% 

Greater  than  330,800 

16.67% 

What  will  be  the  official  Sept.  2009 
national  seasonally-adjusted 
unemployment  rate  (per  US  DoL)? 

(9.4%  in  July;  9.7%  in  August) 

Aug.  10,  2009 

Sep.  30,  2009 

Multiple-choice 

Less  than  9.0% 

16.67% 

9.1  -9.3% 

16.67% 

9.4  -  9.6% 

16.67% 

9.7  -  10.0% 

16.67% 

10.1  -  10.5% 

16.67% 

Greater  than  10.5% 

16.67% 

Will  the  Dow  Jones  Industrial  Average 
(INDU)  close  above  9,400  by  COB  on 
Friday,  Aug.  14,  2009?  (Closed  at  9,370 
on  Aug.  7,  2009) 

Aug.  10,  2009 

Aug.  14,  2009 

Binary 

Yes 

50% 

No 

50% 

On  September  30,  2009,  what  will  the 
Navy's  FYlOenlisted  accession  goal 

be? 

Aug.  17,  2009 

Sep.  30,  2009 

Multiple-choice 

Below  35,750 

33.33% 

Exactly  35,750 

33.33% 

Above  35,750 

33.33% 

Which  series  will  receive  the 

"Outstanding  Drama  Series"  award  at 
the  61st  Primetime  Emmys?  (See 
details  in  Market  Information) 

Aug.  17,  2009 

Sep.  20,  2009 

Multiple-choice 

Big  Love 

14.29% 

Breaking  Bad 

14.29% 

Damages 

14.29% 

Dexter 

14.29% 

House 

14.29% 

Lost 

14.29% 

Mad  Men 

14.29% 

Which  series  will  receive  the 

"Outstanding  Comedy  Series"  award  at 
the  61st  Primetime  Emmys?  (See 
details  in  Market  Information) 

Aug.  17,  2009 

Sep.  20,  2009 

Multiple-choice 

Entourage 

14.29% 

Family  Guy 

14.29% 

Flight  Of  The  Conchords 

14.29% 

How  I  Met  Your  Mother 

14.29% 

The  Office 

14.29% 

30  Rock 

14.29% 

Needs 

14.29% 

Who  will  win  the  2009  NFLopening 
game  between  the  Pittsburgh  Steelers 
and  Tennessee  Titans?  (See  details  in 

Sep.  1,  2009 

Sep.  10,  2009 

Multiple-choice 

Tennessee  Titans 

40.00% 

Washington  Redskins 

59.90% 

Tie 

0.10% 

How  many  Major  League  Baseball 
teams  will  clinch  a  playoff  spot  before 
October  1st,  2009?  (See  details  in 
Market  Information) 

Sep.  8,  2009 

Sep.  30,  2009 

Multiple-choice 

Less  than  4 

20% 

4 

20% 

5 

20% 

6 

20% 

7-8 

20% 

Will  the  FY  2010  Defense 

Appropriation  bill  be  signed  into  law 
before  October  1st,  2009? 

Sep.  8,  2009 

Sep.  30,  2009 

Binary 

Yes 

50% 

No 

50% 

2.  Setting  Initial  Market  Prices 

The  initial  market  prices  for  each  security,  with  one  exception,  were  set  at  1/N, 
where  N  is  the  number  of  possible  outcomes  on  which  traders  may  bid.  The  initial  market 
price  for  predicting  the  winner  in  the  2009  National  Football  League  (NFL)  opening 
game  between  Pittsburgh  Steelers  and  the  Tennessee  Titans  was  set  at  $59.90  for 
Pittsburgh  to  win,  $40.00  for  Tennessee  to  win  and  $0.10  for  a  tie.  This  allowed  for  the 
possibility  of  a  tie,  though  a  tie  is  extremely  unlikely.  We  considered  setting  the  initial 
prices  for  predicting  both  the  Navy  endstrength  and  unemployment  rate  to  reflect  a  bell- 
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shaped  distribution  to  account  for  central  tendency.  However,  the  Inkling  software 
automatically  re-sorts  the  contracts  in  an  ascending  order  according  to  price,  which  could 
confuse  the  initial  securities  listing.  Thus,  we  decided  to  set  the  contract  initial  prices 
equally,  using  1/N,  with  the  NFL  football  game  as  the  only  exception.  This  initial  pricing 
policy  provided  an  incentive  for  traders  to  make  relatively  easy  gains  by  participating 
early  to  make  trades  on  seemingly  over-/under-priced  contracts. 


58 


VII.  ASSESSMENT 


A.  TRADER  PARTICIPATION 

Overall,  32  of  58  potential  traders  registered  for  their  N1  Prediction  Market  Pilot 
account.  Of  those  32,  eight  participants  did  not  make  any  trades,  resulting  in  24  actual 
traders.  Please  refer  to  Table  6  for  the  breakdown  of  actual  and  potential  pilot 
participants. 


Table  6.  N1  Prediction  Market  Pilot  Participants 


Actual 

Non-users 

Potential 

Number  of  NPS  participants 

5 

5 

Number  of  N 104  participants 

8 

18 

Number  of  Non-NPS  and  Non-N104  participants 

11 

35 

Number  of  Potential  Participants  that  did  not 

create  their  accounts 

26 

Number  of  Potential  Participants  that  created 

accounts  but  did  not  trade 

8 

Total 

24 

34 

58 

Initially,  there  appeared  to  be  high  organizational  interest  in  the  prediction  market 
pilot.  However,  the  total  number  of  traders  and  trades  conducted  declined  as  the 
prediction  market  progressed.  Figure  1  provides  a  graphical  view  of  the  pilot  by  question 
and  depicts  the  respective  number  of  traders  and  trades. 
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■  Numberof  Trades 


Figure  1 .  Quantity  of  Traders  and  Trades  by  Question 

B.  QUANTITY  OF  TRADES 

The  number  of  trades  also  declined  rapidly  after  the  first  two  weeks,  as  shown  in 
Figure  2.  Of  all  the  trades  conducted  in  the  52-day  pilot,  56.9%  were  conducted  in  the 
first  two  weeks.  The  decrease  in  trading  could  reflect  traders  believing  the  markets  were 
appropriately  priced  and  had  reached  market  equilibrium.  This,  however,  does  not  seem 
to  be  the  case  because  there  were  many  opportunities  for  a  seemingly  easy  gain  as  new 
securities  were  introduced  using  the  1/N  initial  pricing.  Thus,  some  securities  simply 
were  not  appropriately  priced  from  the  outset  and  traders  did  not  take  advantage  of  these 
opportunities.  Therefore,  we  conclude  the  novelty  had  worn  off  for  the  traders,  and  their 
participation  waned. 
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Figure  2.  Quantity  of  Trades  by  Week  and  Office  Code^^ 


Table  7  depicts  the  quantity  of  trades  per  question  by  organization.  The  first  four 
questions  in  the  pilot  were  germane  to  the  N1  organization.  The  remaining  questions, 
shaded  in  gray,  were  added  for  “fun”  and  were  intended  to  spur  interest  and  enhance 
participation.  Surprisingly,  78%  of  the  trades  were  conducted  on  the  N1  relevant 
questions.  There  was  much  less  trading  on  the  “fun”  questions. 


15  Week  8  consisted  of  only  3  days  since  the  market  concluded  at  the  end  of  the  Fiscal  Year. 
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Table  7.  Quantity  of  Trades  per  Question  by  Organization 


Question 

N104 

NPS 

Other 

Grand  Total 

What  will  be  the  Navy's  endstrength  (for  officers 
and  enlisted  personnel)  for  FY2009?  (See  details 
in  Market  Information) 

37 

21 

12 

70 

What  will  be  the  official  Sept.  2009  national 
seasonally-adjusted  unemployment  rate  (per  US 
Dot)?  (9.4%  in  July;  9.7%  in  August) 

33 

14 

14 

61 

Will  the  Dow  Jones  Industrial  Average  (INDU) 
close  above  9,400  by  COB  on  Friday,  Aug.  14, 

2009?  (Closed  at  9,370  on  Aug.  7,  2009) 

16 

21 

1 

38 

On  September  30,  2009,  what  will  the  Navy's  FYIO 
enlisted  accession  goal  be? 

9 

8 

7 

24 

Which  series  will  receive  the  "Outstanding  Drama 
Series"  award  at  the  61st  Primetime  Emmys? 

(See  details  in  Market  Information) 

2 

5 

5 

12 

Which  series  will  receive  the  "Outstanding 
Comedy  Series"  award  at  the  61st  Primetime 
Emmys?  (See  details  in  Market  Information) 

3 

6 

4 

13 

Who  will  win  the  2009  NFL  opening  game 
between  the  Pittsburgh  Steelers  and  Tennessee 
Titans?  (See  details  in  Market  Information) 

1 

2 

2 

5 

Flow  many  Major  League  Baseball  teams  will 
clinch  a  playoff  spot  before  October  1st,  2009? 

(See  details  in  Market  Information) 

1 

11 

2 

14 

Will  the  FY  2010  Defense  Appropriation  bill  be 
signed  into  law  before  October  1st,  2009? 

2 

3 

4 

9 

Grand  Total 

104 

91 

51 

246 

C.  PILOT  PREDICTION  MARKET  ASSESSMENT 

The  reduction  in  trading  volume  may  reflect  the  topics  or  the  timing  of  their 
introduction.  We  predicted  the  first  two  “fun”  questions,  regarding  the  Primetime 
Emmy’s  for  Drama  and  Comedy  series,  would  have  wide  appeal  and  elicit  participation. 
That  proved  not  to  be  the  case.  These  questions  were  introduced  at  the  same  time  as  the 
Navy’s  FYIO  enlisted  accession  goal  question.  The  number  of  trades  on  the  Navy’s 
enlisted  accession  goal  was  nearly  equal  to  the  number  of  trades  on  the  two  Emmy 
questions. 

We  thought  introducing  a  question  about  the  2009  NEE  opening  game  between 
the  Tennessee  Titans  and  Pittsburgh  Steelers  would  be  an  engaging  question,  considering 
the  general  enthusiasm  for  the  beginning  of  the  NFL  season  and  the  teams  involved. 
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However,  this  question  was  the  least  traded  question  in  the  pilot  market.  The  Pittsburgh 
Steelers  were  favored  to  win  by  more  than  five  points.  Perhaps  the  traders  thought  the 
contract  was  already  appropriately  priced.  Nonetheless,  we  suspect  that  interest  in  the 
prediction  market  had  waned  by  this  point  and  traders  simply  were  not  actively  trading 
anymore. 

Furthermore,  trading  on  this  question  remained  open  throughout  the  game  and  for 
one  hour  following  the  game;  leaving  the  market  open  provided  an  opportunity  for 
guaranteed  profits,  as  the  price  for  Pittsburgh  remained  at  $60.44  even  though  the  payoff 
would  be  $100  per  share  for  a  Pittsburgh  victory.  Although  the  football  game  had  four 
lead  changes  and  ultimately  was  decided  in  overtime,  no  traders  conducted  in-game 
trades;  moreover,  no  traders  exploited  the  guaranteed  after-game  profit  potential.  In  the 
traders’  defense,  the  game  ended  around  11:30  pm  EDT.  Therefore,  those  traders  who 
watched  the  game  likely  went  to  sleep  at  the  conclusion  of  the  game  instead  of  thinking 
about  checking  to  see  how  well  they  did  in  the  pilot  prediction  market  or  exploring  an 
opportunity  to  make  an  ex-post  trade  for  guaranteed  profit. 

The  other  two  “fun”  questions  also  had  poor  trading  volume.  Taken  together, 
introducing  “fun”  questions  certainly  did  not  elicit  a  cycle  of  engagement,  participation, 
and  trading  as  anticipated.  Perhaps  there  was  a  mismatch  in  NPS  researcher  and  N1 
member  interests.  N1  participation  may  have  been  better  if  we  had  predetermined  their 
interests  to  ask  questions  they  think  are  fun. 

Data  shows  that  participation  was  low,  especially  by  the  Non-NPS  and  Non-N104 
participants,  where  only  11  of  35  potential  participants  actually  made  trades.  Because 
trading  decreased  significantly  after  the  first  two  weeks,  we  suspect  that  interest  in  the 
prediction  market  pilot  quickly  wore  off  once  the  novelty  was  over.  Moreover,  the  traders 
had  no  real  incentive  to  participate,  and  we  believe  this  limited  interest.  Prediction 
markets  simply  will  not  work  without  active  participation. 


63 


D.  LESSONS  LEARNED  WITH  SUBSTANTIATING  POST-PREDICTION 

MARKET  SURVEY  RESULTS 

We  conducted  two  anonymous  post-market  surveys:  a  “user”  survey  for  those 
who  registered  and  conducted  at  least  one  trade,  and  a  “non-user”  survey  for  those  who 
did  not  participate.  These  surveys  featured  many  multiple-choice  questions.  Respondents 
were  asked  to  select  all  applicable  responses.  The  questions  and  potential  responses  for 
the  two  surveys  are  provided  in  Appendices  A  and  B. 

The  “user”  survey  asked  active  participants  about  their  motivations  to  initially 
trade  and  sustain  trading  past  the  first  few  weeks,  as  well  as  their  thoughts  regarding 
implementation,  operation  and  outcome  of  the  prediction  market  experiment.  Overall,  12 
of  24  potential  respondents  completed  the  “user”  survey;  3  of  these  12  respondents  were 
from  NPS.  The  “non-user”  survey  was  intended  for  those  who  were  invited  to  participate 
in  the  experiment  but  chose  not  to  do  so.  We  wanted  to  find  out  why  they  chose  not  to 
participate  and  what  incentives  would  entice  them  to  participate.  In  total,  6  of  34  potential 
respondents  completed  the  “non-user”  survey.  We  will  list  the  lessons  learned  from  this 
pilot  prediction  market. 

I.  Participation  is  Critical  for  a  Prediction  Market  to  be  Efficient. 

Furthermore,  Incentives  are  Necessary  for  People  to  Participate. 

The  active-traders  were  initially  highly  motivated  to  participate  due  to  their 
intrigue  in  prediction  markets.  Additionally,  6  of  12  traders  claimed  a  factor  in  their 
decision  to  initially  participate  was  that  the  questions  were  both  relevant  and  interesting. 
See  Figure  3. 
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What  factors  influenced  you  to  initially  participate?  (Please  select  all 

that  apply) 
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Participation  Factors 


Figure  3.  Active  User  Initial  Participation  Factors 


The  motivations  for  traders  to  participate  past  the  first  few  weeks  were  slightly 
different  (see  Figure  4).  The  top  factors  were  still  intrigue  in  prediction  markets  and  the 
draw  of  relevant  and  interesting  questions.  However,  the  intrigue  dropped  significantly 
from  11  of  12  to  7  of  12  traders  indicating  their  intrigue  in  prediction  markets  was  a 
factor  in  participation.  Interestingly,  6  of  12  participants  viewed  the  relevant  and 
interesting  questions  as  a  factor  for  their  continued  participation  past  the  first  few  weeks. 
Another  item  of  interest  is  that  participants  listed  competition  as  an  increased  factor  for 
their  continued  participation.  Nevertheless,  the  precipitous  drop  in  trading  volume  after 
the  first  few  weeks  suggests  that  interest  in  competition  waned.  One  would  expect  trading 
volume  to  increase  if  a  competitive  spirit  had  emerged  and  the  traders  did  not  believe  the 
markets  were  appropriately  priced. 
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What  factors  influenced  your  continued  participation  past  the  first 
few  weeks?  (Please  select  all  that  apply) 

Sample  size  =  12 
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Figure  4.  Factors  for  Active  User  Continued  Participation 


A  prediction  market  would  work  best  if  every  organizational  member  were  a  self- 
selected  participant  who  would  work  to  discover  and  share  his  or  her  information.  If  this 
were  the  case,  incentives  would  not  be  necessary.  The  reality  is  that  most  people  will  not 
do  this.  It  is  obvious  that  monetary  gain  provides  a  natural  participatory  motivation  in 
real  money  markets.  In  play  money  markets,  other  participation  incentives  are  likely 
needed  to  attract  and  sustain  participation. 

Figure  5  shows  the  factors  leading  to  declining  participation  past  the  first  few 
weeks  of  the  pilot  prediction  market.  Four  of  12  active  participants  indicated  that  either 
lack  of  incentive  or  time  caused  their  trading  volume  to  decline  after  the  first  few  weeks. 
Furthermore,  only  1  of  the  12  attributed  their  trading  volume  decrease  to  a  belief  that 
market  questions  were  appropriately  priced  and  had  reached  equilibrium.  Thus,  evidence 
suggests  that  incentives  are  necessary  for  traders  to  overcome  time  constraints  to  register 
and  sustain  participation. 


66 


If  you  made  early  trades  but  reduced  your  trading  activity,  what  if 
anything,  limited  your  participation?  (Please  select  all  that  apply) 

Sample  size  =  12 
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Factors  that  Limited  Participation 


Figure  5.  Active  User  Reasons  for  Decline  in  Trading  Volume  Past  the  First  few 

Weeks 

Given  our  pre-survey  belief  that  incentives  play  a  role  in  participation,  we  asked  a 
question  in  both  the  “user”  and  “non-user”  survey  on  what  incentives  would  motivate 
traders  to  participate  in  a  future  Navy-sponsored  prediction  market.  We  provided  a  list  of 
tangible  and  intangible  incentives  for  respondents  to  choose  from  and  gave  them  the 
opportunity  to  enter  “Other”  preferences  in  a  free  text  area.  Figure  6  shows  the  results  for 
active-users.  They  displayed  an  affinity  for  tangible  rewards,  such  as  cash  prizes  or  an 
iPod.  Interestingly,  2  of  12  respondents  indicated  they  would  participate  with  no 
incentives.  In  addition,  3  of  6  “Other”  responses  indicated  that  the  subjects  were  happy  to 
participate  with  no  incentives  but  their  participation  may  have  increased  if  a  tangible 
prize  were  offered.  The  remaining  three  textual  responses  were  singular  votes  for  tangible 
prizes,  special  liberty  and  no  incentives  required. 
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What  incentives  would  cause  you  to  participate  in  a  future 
DoD/  Navy  prediction  market?  (Please  select  all  that  apply) 

Sample  size  =  12 
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Participation  Incentives 


Figure  6.  Active-User  Incentive  Preferences 


Figure  7  presents  the  non-user  responses  to  the  same  question.  One  non-user 
indicated  that  no  incentives  would  be  necessary  for  him/her  to  participate  in  a  future 
Navy  prediction  market.  The  non-user  responses  to  the  incentive  question  were 
dominated  by  the  “Other”  response.  One  non-user  indicated  he/she  would  participate  if 
he  were  given  paid  time  off  from  work — even  just  a  few  hours.  The  remaining 
respondents  who  selected  “Other”  did  not  answer  the  question  directly  and  expressed 
frustration  in  the  free  text  area  regarding  technical  issues,  account  setup,  uninteresting 
questions  and  ambiguity  with  the  concept.  Interestingly  though,  none  of  the  non-users 
indicated  they  would  not  participate  in  a  future  prediction  market. 
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What  incentives  would  cause  you  to  participate  in  a  future 
DoD/ Navy  prediction  market?  (Please  select  all  that  apply) 

Sample  size  =  6 
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Figure  7.  Non-User  Incentive  Preferences 


Despite  the  explicit  preference  for  tangible  prizes  from  both  the  active-  and  non¬ 
users,  there  are  several  reasons  to  believe  that  social  rewards  may  induce  participation 
more  effectively.  The  primary  reason  is  that  expected  monetary  or  other  rewards  have  a 
low  relative  value  compared  to  the  time  and  energy  expended  to  discover  new 
information  and  make  trades  (Cowgill,  2007).  Moreover,  neither  profit  maximizing 
companies  nor  thrifty  government  organizations  expend  the  resources  to  make  these 
tangible  rewards  worth  their  employees’  time  to  fully  engage  in  the  prediction  market  and 
meet  the  requirements  of  their  individual  jobs.  An  organization  may  be  able  to  foster 
competition  between  employees  or  organizational  divisions  to  create  a  culture  that  values 
top  individual  and  divisional  trader  bragging  rights.  Organizations  may  be  able  to  curtail 
tangible  incentives  once  this  type  of  culture  is  in  place.  For  suggestions  on  how  to  foster 
this  type  of  culture.  Bo  Cowgill  provides  excellent  insight  into  how  Google  was  able  to 
develop  a  culture  in  which  employees  value  their  reputation  as  top  traders  more  than 

monetary  rewards. 


This  is  a  frequent  topic  for  Bo  Cowgill.  One  transcript  regarding  this  subject  may  be  found  at: 
http://www.midasoracle.Org/2007/04/l  1/how-prediction-exchanges-can-best-encourage- 
participation/#comments . 
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2.  Participant  Selection  should  be  more  Inclusive  than  Exclusive. 

A  diverse  and  broad  pool  of  prediction  market  participants  is  optimal.  As 
discussed  many  times  in  this  paper,  it  is  impossible  for  the  organization  to  determine 
exactly  who  has  useful  knowledge.  The  active-users  were  more  inclusive  than  exclusive 
when  expressing  their  opinion  of  what  groups  of  people  should  be  included  in  future 
Navy  N1  prediction  markets.  When  asked,  “What  group(s)  of  Navy  personnel  should  be 
included  in  future  N1  prediction  markets  to  help  better  aggregate  force- structure 
information,”  respondents  overwhelmingly  agreed  that  recruiters,  manpower  analysts, 
and  budget  analysts  should  be  involved.  The  single  “Other”  response  was  a  free  text 
opinion  that  “any  and  all  people  connected  with  Navy  MPT&E”  should  participate.  Refer 
to  Figure  8. 


Whatgroup(s)  of  Navy  personnel  should  be  included  in  future  N1 
prediction  markets  to  help  better  aggregate  force -structure 
information?  (Please  select  all  that  apply) 

Sample  size  =  12 
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N1  Functional  Groups 


Figure  8.  Active-User  Opinion  Regarding  Which  Groups  of  N1  Members  should  be 

Included  in  Future  Navy  N1  Prediction  Markets 


How  does  an  organization  know  who  or  how  many  people  should  participate?  It  is 
easy  to  say  that  an  organization  should  be  more  inclusive  than  exclusive.  However,  as  a 
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practical  consideration  it  is  clear  that  not  every  individual  needs  to  participate  in  a 
prediction  market,  especially  in  very  large  organizations  such  as  the  Navy.  The  location 
of  that  dividing  line  is  an  open  question. 

There  is  not  a  specific  ratio  of  people  who  should  (or  should  not)  participate. 
Rather  the  organization  must  weigh  the  costs  and  expected  benefits  of  implementing  the 
prediction  market.  Using  this  as  a  guide,  the  costs  should  not  exceed  the  expected  benefit. 
The  decision  of  whether  to  conduct  a  prediction  market  and  who  should  participate  is 
driven  by  these  factors: 

•  Fixed  Costs: 

o  Management  and  support  costs 
o  Cost  of  tangible  incentives 

•  Variable  Costs: 

o  Participant’s  time  spent  training  (this  will  vary  by  labor  rate  and  the 

length  of  the  training  program) 

o  Participant’s  time  spent  trading  (this  will  vary  by  labor  rate  and  time 
spent  trading) 

■  One  way  to  limit  this  cost  is  to  restrict  trading  to  specific  days  or 
times 

o  Interface  costs  if  using  a  commercial  service  (usually  a  per  user  fee  per 
month) 

3.  Thorough  Group  Training  should  he  Provided  to  Potential  Prediction 

Market  Participants  to  Explain  the  Prediction  Market  Concept  and 
its  Organizational  Purpose  and  Intent. 

In  the  post-market  survey,  non-users  were  asked  what  factors  affected  their 
decision  not  to  participate  in  the  pilot  market.  Refer  to  Figure  9  for  potential  responses 
and  results.  Two  of  6  respondents  indicated  they  did  not  understand  the  concept  or  intent 
or  did  not  participate  because  of  technical  issues.  A  group  training  session  could  also 
include  the  account  setup  for  the  chosen  prediction  market  interface.  The  group  training 
would  provide  fewer  excuses  for  people  not  to  participate  with  the  participants’  accounts 
setup  and  any  technological  hurdles  addressed. 
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What  affected  your  decision  to  not  participate?  (Please  select  all 

that  apply) 

Sample  size  =  6 


c 

o 

u 


in 

c 

o 

Q. 

l/l 


tt) 

a: 


Non -User  Participation  Factors 


Figure  9.  Non-user  Participation  Factors 


The  training  would  also  provide  an  opportunity  for  senior  leaders  to  sponsor  the 
prediction  market  concept.  The  N1  leadership  may  have  inadvertently  undermined  the 
prediction  market  experiment  by  simply  forwarding  the  introductory  emails  from  NPS 
students  vice  sending  the  emails  as  if  coming  directly  from  an  N 1  senior  leader.  In  fact,  3 
of  6  non-users  indicated  they  thought  the  prediction  market  experiment  was  simply  an 
NPS  student  thesis  project,  whereas  none  of  the  active  users  felt  this  way.  Conversely,  10 
of  12  active  users  thought  the  experiment  was  a  joint  N1  and  NPS  thesis  project;  the 
remaining  two  perceived  the  experiment  as  an  N1  initiative.  Our  surveys  showed  that 
executive  sponsorship  is  of  the  utmost  importance  to  ensure  prediction  market 
participation. 

The  training  should  also  include  a  tutorial  regarding  trading  strategies.  The  goal  is 
to  educate  participants  about  long  and  short  selling  strategies  along  with  their  advantages 
and  disadvantages.  The  Inkling  interface  made  short  selling  extremely  user  friendly,  such 
that  traders  may  not  have  fully  understood  the  concept  even  though  they  thought  they  did. 
However,  nine  of  12  active  users  indicated  that  they  understood  the  concept  of  short 
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selling.  We  cannot  determine  whether  they  fully  understood  the  concept  or  if  the  Inkling 
interface  made  short  selling  so  easy  that  participants  thought  they  understood  it. 

4.  When  Attempting  to  Forecast  an  Organizational  Metric,  Develop  a 
Complete  Understanding  of  what  is  Being  Measured  and  from  Where 
the  Data/Results  Come. 

As  previously  discussed,  N1  initially  wanted  to  forecast  the  Nuke  Zone  A 
retention  rate.  With  minimal  research,  we  discovered  that  the  candidate  question  would 
not  be  very  useful  in  a  prediction  market  at  that  time,  as  the  Navy  had  temporarily 
suspended  re-enlistment  bonuses. 

5.  Properly  Phrasing  Prediction  Market  Questions  is  a  Challenging  Task 
that  Requires  Great  Care. 

It  is  very  easy  to  introduce  ambiguity  in  a  question  with  haphazard  phrasing.  For 
example,  we  published  a  market  with  the  following  phrasing:  Will  the  Dow  Jones 
Industrial  Average  (INDU)  close  above  9,400  ^  COB  on  Friday,  Aug.  14,  2009?  (It  had 
closed  at  9,370  on  Aug.  7,  2009  when  the  weeklong  market  was  introduced).  Instead,  we 
meant  to  ask  the  following:  Will  the  Dow  Jones  Industrial  Average  close  above  9,400  on 
Friday,  August  14,  2009,  at  COB?  The  published  phrasing  provided  an  opportunity  for 
the  Dow  Jones  Industrial  Average  to  eclipse  9,400  each  day  up  to  August  14,  2009. 
Hence,  we  were  forced  to  closely  monitor  the  conclusion  of  the  stock  market  each  day  in 
the  event  the  Dow  Jones  Industrial  Average  closed  above  9,400.  If  this  condition  were 
met,  we  would  have  preemptively  closed  and  paid  out  the  market  before  the  intended 
closing  date  of  August  14,  2009. 

We  faced  two  additional  challenges  with  phrasing.  One  challenge  was  jargon 
ambiguity.  The  N1  jargon  was  the  Nuke  Zone  A  retention  rate.  We  simply  did  not 
understand  the  rules  for  determining  whether  a  Nuke  was  in  Zone  A  (or  another  zone). 
Secondly,  the  Inkling  interface  limited  the  question  length  to  140  characters  for  Twitter 
integration.  This  140  character  constraint  made  it  very  difficult  for  us  to  ask  complicated 
questions.  We  generally  used  the  Market  Information  text  box  to  provide  supplemental 
details.  Unfortunately,  the  Market  Information  text  box  appeared  at  the  bottom  of  the 

webpage  and  may  not  have  been  obvious  to  participants. 
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Despite  these  challenges,  we  were  pleased  to  find  out  the  drop  off  in  participation 
was  not  because  of  poorly  phrased  questions.  In  fact,  11  of  12  active  users  felt  the 
prediction  market  questions  were  somewhat  or  very  clear. 

6.  Running  a  Prediction  Market  takes  Time  and  Effort. 

This  may  seem  an  obvious  statement  but  the  point  should  not  be  taken  lightly. 
Our  recommendation  is  to  outsource  the  initial  market  setup.  There  are  many  decisions  in 
the  initial  setup  that  affect  the  market’s  success.  After  the  market  is  established,  one 
person  could  easily  manage  accounts  and  technical  issues  as  a  collateral  duty.  However, 
it  would  be  unwise  to  task  one  person  to  generate  interesting,  yet  unambiguous  questions. 

Perhaps  the  most  difficult  task  was  to  generate,  formulate,  and  vet  questions  for 
all  possible  trader  interpretations.  It  would  be  very  difficult  for  a  single  person  to 
consider  how  different  participants  may  interpret  the  prediction  market  questions.  Even 
with  three  to  five  people  reviewing  each  question  before  publication,  we  still  ended  up 
with  at  least  one  poorly  worded  question.  We  recommend  that  a  group  of  approximately 
three  to  five  middle-management  personnel  perform  this  task.  This  group  should  publish 
questions  and  monitor  trader  comments  for  possible  confusion.  If  necessary,  they  should 
suspend  trading  and  refund  participants  to  pre-trade  levels.  In  addition,  the  members  of 
this  group  need  to  ensure  they  fully  understand  how  the  outcome  is  measured  and 
whether  the  pay-off  is  clear. 
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VIII.  NAVY-SPECIFIC  CONSIDERATIONS  FOR  PREDICTION 

MARKET  UTILIZATION 

Navy  decision  makers  need  all  available  useful  information  to  make  informed 
decisions.  An  efficient  and  effective  information  aggregation  tool  may  prove  invaluable 
to  Navy  decision  makers  considering  the  gravity  of  the  decisions  they  must  make.  It  is 
important,  however,  to  consider  both  the  benefits  and  the  limitations  of  prediction 
markets  that  most-directly  affect  Navy  utilization  of  prediction  markets. 

A.  BENEFITS 

Generally,  the  prediction  market  benefits  and  limitations  discussed  in  Chapter  III 
hold  true  for  Navy  prediction  markets.  The  dynamic  nature  and  potential  for  accessing 
the  collective  organizational  information  are  among  the  most  important  and  desirable 
benefits  of  prediction  markets. 

The  dynamic  nature  of  prediction  markets  can  help  Navy  decision  makers 
overcome  common  organizational  challenges.  For  instance,  in  some  Navy  commands, 
decision  makers  do  not  have  adequate  access  to  much  of  the  organization’s  existing 
knowledge.  Moreover,  they  simply  may  not  know  it  exists,  or  may  not  know  whom  to  ask 
for  the  information  they  seek.  The  Navy  is  too  large  an  organization  and  too  complex  for 
any  single  individual  to  fully  grasp  the  details  and  inner  workings  of  each  level  of 
operation.  This  adverse  characteristic  unfortunately  translates  down  to  the  command  level 
as  well.  For  instance,  when  tracking  progress  on  an  acquisition  program,  such  as  the  Joint 
Strike  Fighter,  no  one  individual  can  identify,  understand,  and  maintain  every  detail 
associated  with  that  program.  Therefore,  middle-level  managers  collect  and  manage 
information  in  a  departmentalized  way.  However,  information  often  is  lost  in  translation 
and  transmission  upward  through  chains-of-command.  Prediction  markets  would  allow 
Navy  decision  makers  to  easily  query  the  whole  organization  to  aggregate  the  specific 
information  spread  across  the  entire  organization. 

Another  challenge  Navy  decision  makers  face  is  that  some  organizational 
information  is  lost  over  time.  Information  can  be  lost  because  some  people  do  not  have 
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the  incentive  or  wherewithal  to  reveal  it  to  appropriate  decision  makers;  additionally, 
tacit  knowledge  is  almost  certainly  lost  as  military  members  continuously  rotate  in  and 
out  of  Navy  commands.  Easily  accessible  prediction  markets  can  provide  a  means  for 
retrieving  such  information  that  normally  would  be  lost. 

Prediction  markets  may  provide  a  further  benefit  to  the  Navy  because  of  their 
ability  to  aggregate  useful  information  in  an  anonymous  manner.  It  is  always  easy  for 
employees  to  trumpet  pleasant  information  to  their  superiors,  but  many  employees, 
especially  middle  management,  find  it  difficult  to  share  negative  information,  resulting  in 
uninformed  senior  leadership.  Anonymous  prediction  markets  offer  a  solution  for  sharing 
unpleasant  information  and  removing  the  middle  management  bias  by  giving  lower-level 
individuals  a  mechanism  to  share  their  insight  and  perspective,  without  fear  of  middle 
management  reprisal. 

B.  LIMITATIONS 

The  Navy  also  must  consider  prediction  market  limitations  to  make  a  sound 
judgment  on  whether  and  when  to  utilize  prediction  markets.  We  believe  participatory 
challenges  and  organizational/cultural  barriers  are  the  limitations  most  likely  to  affect  the 
success  of  Navy-sponsored  prediction  markets.  These,  in  turn,  would  have  implications 
on  the  markets  that  can  be  used.  Each  challenge  is  discussed  below. 

I.  Participatory  Challenges 

We  believe  there  are  three  primary  reasons  the  Navy  will  face  participatory 

challenges  when  utilizing  prediction  markets.  First,  the  Navy’s  prediction  market  claims 

will  be  highly  specialized — such  as  claims  on  endstrength,  re-enlistment  bonuses, 

recruiting  goals  and  other  issues  that  are  not  common  to  the  layperson — and  they  will 

require  a  degree  of  Navy-specific  knowledge  for  individuals  to  trade  with  any  expertise. 

As  a  result,  most  potential  participants  may  feel  disadvantaged  in  a  given  market  since 

they  believe  other  traders  possess  either  privileged  information  or  greater  specialization, 

while  they  themselves  do  not.  Second,  the  Navy  is  limited  in  the  rewards  it  may  offer 

traders  for  their  participation.  Various  legal  and  ethical  barriers  may  preclude  the  Navy 

from  using  monetary  or  nonmonetary  compensation;  yet,  incentives  are  essential  for 
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ensuring  market  participants  remain  actively  engaged.  Finally,  traders  may  believe  that 
their  participation  is  futile  or  irrelevant  since  market  parameters  can  easily  change. 
Decision  makers  can  affect  prediction  markets  by  changing  policies  or  conditions  that  are 
directly  associated  with  a  given  market;  when  this  occurs,  trader  positions  can  be 
undermined  within  the  affected  market. 

a.  Perceived  Disadvantage 

As  mentioned  above.  Navy  prediction  markets  often  deal  with  very 
specific  items  of  concern.  As  such,  traders  may  need  to  possess  relevant  specialized 
information.  Additionally,  some  traders  may  have  unique  access  to  critical  or  sensitive 
information  relative  to  a  prediction  market,  and  some  may  have  earlier  access  than  others 
to  market  information. 

Although,  the  nature  of  prediction  markets  requires  a  degree  of  insider 
trading  to  reveal  information,  the  detriment  comes  when  only  a  few  individuals  have 
insight  to  the  relevant  market  information.  When  this  occurs,  the  majority  of  the 
participation  pool  is  considered  uninformed,  and  the  informed  traders  can  exploit  their 
insider  positions.  Insider  trading  is  outlawed  in  public  stock  markets  to  protect  non¬ 
insiders.  Yet,  it  is  difficult  to  make  similar  regulations  for  prediction  markets. 

Participants  believing  others  have  a  significant  advantage  (due  to  access  to 
information)  may  avoid  trading  because  of  their  perceived  position  of  inferiority  within  a 
given  market.  As  such,  certain  Navy  markets — such  as  those  pertaining  to  recruiting 
goals  or  endstrength — will  likely  have  a  very  thin  and  specific  participant  pool. 

One  such  case  occurred  within  our  prediction  market  experiment 
involving  the  Navy  endstrength.  Two  traders  from  N104  posted  weekly  updates  of 
current  Navy  force  strength  levels.  This  led  some  traders  to  believe  that  these  individuals, 
if  not  others,  had  early  access  to  critical  information.  Moreover,  one  NPS  trader  explicitly 
indicated  that  he  felt  he  was  at  a  significant  disadvantage  because  other  people  had  this 
early  access  to  the  current  force  numbers.  It  so  happens  that  this  was  the  most-heavily 
traded  market  in  terms  of  total  active  traders  and  total  trades.  This  may  reflect  that  this 
particular  market  was  the  first  market  introduced  to  all  our  participants,  so  it  was  up  and 
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running  when  traders  were  still  active.  Of  note,  roughly  50  of  70  total  trades  within  the 
endstrength  market  occurred  after  the  N104  weekly  forecast  updates  first  began. 
Therefore,  had  other  participants  drawn  the  same  conclusion  as  the  NFS  trader,  this 
particular  market  would  have  experienced  significantly  less  trading  activity  and 
participation. 


b.  The  Navy  is  Limited  in  the  Incentives  it  can  Offer 

Prediction  market  participation  often  hinges  directly  on  participation 
incentives.  Because  legal  restrictions  and  concerns  over  public  perception  likely  preclude 
the  Navy  from  using  real  money  prediction  markets,  it  will  have  to  incentivize 
participation  while  using  play  money  markets.  In  addition,  it  is  unlikely  that  the  Navy 
could  elicit  participation  via  certain  hard  incentives,  such  as  monetary  rewards.  Such 
incentives  might  raise  public  concerns  on  the  proper  and  ethical  use  of  government 
money.  A  secondary  concern  may  be  that  some  cash  rewards  could  shift  Navy  prediction 
market  participants’  loyalties  and  focus  them  away  from  their  primary  duties  and  toward 
their  participation  and  engagement  in  Navy  prediction  markets.  This  last  concern  is 
explained  more  fully  below  in  the  section  discussing  organizational/cultural  barriers. 

c.  Parameters  May  Change 

It  is  important  to  note  that  managerial  decisions  can  drastically  affect  an 
open  market  regardless  of  whether  such  decisions  depend  on  that  market’s  current 
forecasting  assessment.  Herein  lies  the  dilemma:  organizational  leaders  want  to  use 
prediction  markets  as  a  forecasting  tool  to  enhance  their  decision-making  process; 
however,  actions  taken  or  decisions  made  by  these  leaders  can  significantly  affect  open 
markets  and  their  associated  assessments,  potentially  undermining  trader  positions.  Thus, 
some  traders  may  be  less  inclined  to  participate  in  markets  they  believe  to  be  overly 
susceptible  to  influence  through  management  decisions.  As  a  result,  participation  may 
wane  in  certain  Navy  markets. 

For  instance,  imagine  the  Navy  wishes  to  establish  a  market  to  determine 
if  it  will  meet  its  recruiting  goal  in  a  period  when  the  economy  is  growing.  When  the 

economy  is  growing,  the  Navy  has  a  more  difficult  time  reaching  its  recruiting  goals. 
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Nevertheless,  the  Navy  typically  meets  its  recruiting  goals  by  making  appropriate 
management  decisions.  Some  common  decisions  are  to  raise  enlistment  bonuses,  lower 
Armed  Forces  Vocational  Aptitude  Battery  (ASVAB)  score  requirements,  issue  more 
conduct  waivers  (allowing  those  with  a  criminal  history  to  enlist),  or  raise  the  recruit 
maximum  age  requirement.  If  Navy  decision  makers  perceive  they  are  not  reaching 
recruiting  goals,  they  would  make  necessary  changes  to  enlistment  incentives  or 
standards,  changing  the  parameters  for  any  prediction  market. 

This  may  limit  participation  for  two  reasons.  First,  the  likelihood  that  the 
Navy  will  do  what  it  takes  to  meet  its  goals  (for  recruiting  or  retention)  could  make  a 
market  uninteresting  to  some  potential  traders.  On  the  other  hand,  some  may  invest  in  the 
prediction  market  based  on  that  belief.  One  case  from  our  prediction  market  experiment 
highlights  this.  Numerous  traders  participated  in  the  market:  “What  will  be  the  Navy’s 
endstrength  (for  officers  and  enlisted  personnel)  for  FY2009?”  Although  only  one  trader 
offered  a  reason  for  conducting  trades  in  the  manner  that  he/she  did,  that  one  trader 
stated:  “Meeting  endstrength  is  a  high  priority  —  the  Navy  makes  [sic]  it  happen.”  In  this 
instance,  it  is  clear  that  the  trader  believed  the  Navy  would  take  action(s)  to  ensure  the 
established  endstrength  goal  was  met,  and  therefore  conducted  a  market  trade  as  a  result 
of  this  belief. 

Second,  participation  may  be  limited  because  the  changed  parameters  may 
undermine  traders’  positions.  For  example,  a  trader  may  have  believe  that  the  economy  is 
going  to  be  stronger  (and  thus  the  recruiting  environment  will  be  more  difficult)  than 
most  people  think,  but  they  may  be  hesitant  to  invest  in  an  outcome  based  on  this  belief 
knowing  that  the  Navy  may  change  the  parameters  (e.g.,  raise  enlistment  bonuses)  to 
meet  the  end-goal.  Many  individuals  in  the  Navy  believe  the  Navy  will  act  to  ensure  they 
meet  established  goals.  Therefore,  potential  traders  may  be  less  likely  to  participate  in  a 
market  associated  with  a  goal  open  to  managerial  influence.  This  concern  does  not  apply 
to  all  outcomes,  but  it  applies  to  outcomes  such  as  recruiting  and  retention. 

In  markets  with  outcomes  for  which  the  Navy  can  change  the  underlying 

parameters,  incorporating  specific  market  rules  to  accommodate  unexpected  managerial 

parameter  changes  may  alleviate  traders’  concerns.  One  common  rule  pays  traders  at  the 
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time  of  any  major  policy  change  that  could  affect  the  prediction  market  outcome.  Such  a 
rule  must  be  in  place  and  communicated  clearly  to  all  potential  participants  prior  to  and 
during  a  given  market’s  open  period.  This  ensures  that  all  participants  are  aware  of  the 
potential  for  change  and  that  an  identifiable  contingency  is  in  place  for  such  change. 

As  a  consequence  of  potentially  limited  participation  from  these  problems, 

the  Navy  is  susceptible  to  thin  markets.  If  the  potential  participant  pool  is  small,  an 

automated  market  maker  is  necessary  to  promote  liquidity  and  avoid  high  person-to- 

person  bid/ask  spreads.  As  discussed  earlier,  thin  markets  are  not  a  preferred  condition 

for  prediction  markets.  Moreover,  automated  market  makers  are  not  necessarily  desirable, 

1  8 

as  they  inject  an  unwanted  element  of  artificiality  into  the  markets. 

To  conclude,  participatory  challenges  will  affect  the  Navy’s  ability  to 
usefully  employ  some  prediction  markets.  These  challenges  include  each  of  the 
following:  1)  some  traders  perceive  themselves  to  be  at  a  disadvantage  relative  to  other 
traders  who  may  possess  privileged  information,  2)  the  Navy  is  limited  in  the  incentives 
it  can  offer,  and  3)  prediction  market  parameters  may  change.  Each  of  these  issues  could 
affect  a  potential  trader’s  desire  to  participate  in  a  given  prediction  market,  and  each  can 
ultimately  cause  overall  participation  levels  to  decrease.  Furthermore,  each  of  the 
participatory  challenges  can  lead  to  thinner  markets.  As  a  result  of  thin  markets,  the  Navy 
likely  would  have  to  use  some  market  maker  to  promote  liquidity  throughout  the  markets. 
Unfortunately,  market  makers  can  inject  artificiality  into  markets. 

2.  Organizational  /  Cultural  Barriers 

In  addition  to  participatory  challenges,  the  Navy  may  have  great  barriers  to 
overcome  in  terms  of  its  organizational  structure  and  culture.  Strong-rooted,  hierarchical 
organizations  such  as  the  Navy  may  be  less  inclined  to  adopt  and  embrace  prediction 
markets  without  skepticism  and  resistance  from  many  people  throughout  the  chain-of- 

As  discussed  in  Chapter  V,  automated  market  makers  act  as  a  universal  buyer/seller  to  permit  trades 
to  occur  without  requiring  direct  trader-to-trader  interactions. 

As  mentioned  earlier,  the  way  in  market  makers  conduct  their  calculations  often  is  based  on  the 
market  manager’s  arbitrary  assignment  of  specific  values  within  the  market  maker’s  root  formulas.  As  a 
result,  the  market  maker  artificially  approximates  the  market’s  supply  and  demand  equilibrium,  resulting  in 
a  correlated  security  price — instead  of  the  new  market  price  simply  being  set  by  the  lowest  asking  price. 
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command;  individuals  generally  oppose  change  within  such  organizations  because  they 
routinely  and  frequently  experience  change  with  senior  leaders  rotating  in  and  out  of 
command  positions.  Furthermore,  there  stands  to  be  a  blemish  on  senior  decision  makers’ 
records  if  they  take  acions  that  oppose  fulfilled  beliefs  of  the  collective  prediction  market 
group.  In  other  words,  if  the  group  believes  a  given  event  will  occur,  and  its  belief  is 
contrary  to  the  opinion  of  senior  Navy  leaders,  then  the  senior  leaders  may  appear  inept  if 
the  event  actually  occurs  according  to  the  group’s  forecasted  belief.  Tom  Davenport  of 
Babson  College  elaborates: 

The  barriers  to  adoption  of  prediction  markets  are  primarily  cultural  .... 

Let's  say  that  your  company  runs  a  prediction  market  on  first-year  sales  of 
a  new  product,  and  the  results  come  out  not  so  positively.  Let's  say  that  the 
employees  who  participate  predict  much  lower  sales  than,  say,  the  product 
manager  for  the  new  product,  the  division  president,  and  the  CEO.  .  .  .  The 
crowd  has  made  the  hierarchy  look  bad,  and  the  hierarchy  doesn't 
generally  like  to  look  bad.  (Davenport,  2008) 

Because  of  this  potential  problem,  prediction  markets  likely  will  be  most  effective 
and  welcomed  in  Navy  commands  that  have  particularly  strong  leaders,  who  are  open  to 
change  and  readily  acknowledge  they  are  not  always  capable  of  making  the  best 
decisions  without  the  aid  of  their  subordinates  and  colleagues. 

Aside  from  the  issue  of  professional  image,  other  organizational  barriers  may 
exist.  Some  employees  may  work  in  a  position  that  affords  them  an  opportunity  to  affect 
a  given  prediction  market.  Worse  yet,  some  employees  may  work  in  a  position  that 
allows  them  to  affect  organizational  operations  because  of  personal  interest  in  a  given 
prediction  market.  Individuals  in  this  sort  of  position  likely  have  excellent  insight  to  offer 
the  market  as  a  whole.  However,  the  potential  consequences  of  participation  by  these 
people  raise  more  questions  that  Navy  decision  makers  must  consider  prior  to 
implementing  prediction  markets: 

•  Should  individuals  (or  teams)  with  such  positions  be  permitted  to 
participate  in  the  market? 

•  Should  their  participation  in  the  market  be  limited  or  controlled? 
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•  Is  there  another  way  to  work  their  knowledge  into  the  market  without 
risking  a  conflict  of  interest? 

These  questions  and  challenges  create  a  serious  quandary  for  any  organization, 
but  the  nature  of  the  Navy’s  work  makes  them  ever  more  important  and  challenging. 

C.  WHEN  NOT  TO  USE  PREDICTION  MARKETS 

There  are  several  conditions  for  which  prediction  markets  probably  would  not  add 
any  value  to  current  forecasting  techniques.  The  first  condition  is  when  outcomes  have  a 
high  degree  of  predictability.  There  is  little  advantage  to  using  a  prediction  market  to 
predict  events  that  are  highly  predictable.  The  organization  should  simply  rely  on 
historical  data  and  patterns  to  forecast  these  events.  In  our  market,  one  trader  commented 
on  the  enlisted  accession  goal  question  that  the  Navy  would  do  what  it  takes  to  meet  their 
retention  goal.  He  intuitively  knew  that  they  would  make  a  management  decision  to 
change  parameters  to  meet  the  enlisted  accession  goal. 

Prediction  markets  may  offer  a  benefit  to  the  Navy  when  historical  data  is  limited, 
erratic,  or  unreliable — under  such  conditions,  statistical  and  econometric  forecasting 
methods  generally  cannot  provide  accurate  forecasts. 

The  second  condition  rendering  prediction  markets  less  useful  is  when  there  is 
limited  dispersed  information  to  aggregate  or  market  participants  cannot  retrieve  existing 
information.  The  point  of  a  prediction  market  is  to  aggregate  available  information  from 
amongst  its  participants.  If  information  does  not  exist,  participants  do  not  possess 
relevant  information  or  participants  cannot  retrieve  existing  information  then  the 
prediction  market  cannot  function.  Furthermore,  if  traders  attempted  to  participate  in  a 
market  that  is  incapable  of  aggregating  information  because  of  the  condition  above,  the 
collective  market  assessment  is  uncertain,  making  the  assessment  unfit  for  use  by 
organizational  decision  makers. 

The  third  condition  is  when  there  are  only  a  limited  number  of  knowledgeable 
traders.  There  is  no  need  in  using  a  prediction  market  to  forecast  an  event  when  an 
organization  can  reliably  turn  to  one  or  a  few  members  for  the  information.  One  trader  in 
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our  prediction  market  experiment  had  early  access  to  Navy  endstrength  numbers.  This 
trader  shared  that  information  with  the  rest  of  the  traders  and  updated  the  market  with 
weekly  numbers.  As  previously  mentioned,  other  people  may  have  opted  not  to 
participate  on  this  question  because  they  perceived  this  individual  to  have  insider 
knowledge. 

Prediction  markets  seem  to  be  a  natural  fit  for  determining  Navy  project 
management  and  acquisition  outcomes,  such  as  determining  how  a  given  project’s 
progress  is  tracking  with  respect  to  all  departments,  divisions,  and  subprojects.  For 
example,  “When  will  the  Joint  Strike  Fighter  program  receive  milestone  decision 
authority  approval  to  begin  full  rate  production?”  There  appear  to  be  fewer  instances 
where  prediction  markets  are  appropriate  for  determining  outcomes  in  the  Navy  N1 
domain.  It  seems  N1  should  have  the  technology  to  collect,  aggregate  and  access 
quantifiable  information  on-demand. 

D.  THE  WAY  FORWARD  FOR  NAVY  PREDICTION  MARKET 

APPLICATION 

Prediction  markets  can  be  a  positive  and  powerful  decision-making  resource,  and 
they  can  have  a  place  in  Navy  forecasting — with  strong  leadership  and  a  positive 
organizational  culture  that  embraces  change  and  opportunity.  However,  what  we  have 
seen  to  this  point  is  that  the  challenges  and  limitations  of  prediction  markets  are 
extremely  significant,  and  they  may  be  difficult  to  overcome. 

Should  Navy  leadership  use  prediction  markets  in  a  formal  and  official  manner, 
the  top  levels  of  the  chain-of-command  must  accept,  embrace,  and  encourage  the 
markets.  Moreover,  leadership  must  establish  a  full  training  initiative  to  ensure  all 
members  of  the  organization  understand  the  purpose  of  markets,  as  well  as  their  roles 
within  the  market.  Finally,  Navy  leadership  must  fully  understand  the  intricacies  of 
prediction  markets  to  employ  them  properly.  Lessons  learned  from  this  thesis  highlight 
the  fact  that  it  is  an  extremely  difficult  task  to  understand  what  organizational  issues  or 
concerns  are  best  assessed  by  prediction  markets.  For  instance,  an  organization  may 
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forecast  specific  sensitive  metrics,  such  as  endstrength,  but  prediction  markets  can  be 
complicated  when  forecasting  these  metrics  because  of  the  following  additional 
problems: 

•  Inconsistencies  in  organizational  or  cross-organizational  terminology, 

•  Misunderstandings  or  misperceptions  among  organizational  leadership  or 

members  on  what  truly  is  or  should  be  forecasted,  and 

•  Difficulty  in  pinpointing  the  means  by  which  organizations  officially 

measure  a  given  metric;  it  is  essential  to  ensure  markets  are  appropriately 

closed  and  contract  payouts  are  noncontroversial. 

Based  on  post-market  survey  results  and  lessons  learned  during  this  thesis 
research,  the  following  are  some  manpower  outcomes  (not  considered  before  in  our 
discussions  with  Nl)  that  could  potentially  work  in  a  prediction  market: 

•  Advancement  opportunities; 

•  Attrition  rates — unlike  recruiting  and  retention  outcomes,  attrition  would 

not  likely  be  subject  to  policy  changes; 

•  Whether  the  Navy  will  change  certain  recruiting  and  retention  incentives, 
for  example: 

o  Increase  (or  decrease)  an  enlistment  bonus 

o  Increase  (or  decrease)  SRB’s  for  particular  ratings 

o  Increase  (or  decrease)  the  limit  on  enlistees  without  a  high  school 
diploma 

Note  that  all  of  these  would  avoid  the  problem  of  management  changing  policy  levers, 
which  could  undermine  traders’  positions  in  certain  outcomes. 
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IX.  CONCLUSION  AND  RECOMMENDATIONS 


Navy  N1  regularly  forecasts  re-enlistment  rates,  endstrength,  and  many  other 
force-shaping  factors  as  an  input  into  their  resource  allocation  decision-making  process. 
In  an  effort  to  improve  upon  the  force-shaping  decision-making  process,  N1  has  shown 
interest  in  using  prediction  markets  to  complement  or  replace  alternative  methods  for 
forecasting  various  Navy  force-shaping  elements.  We  conducted  a  pilot  prediction  market 
with  N1  personnel  as  the  participants.  We  subsequently  conducted  two  anonymous  post¬ 
prediction  market  surveys.  A  “user”  survey  for  those  who  registered  and  had  at  least  one 
trade  and  a  “non-user”  survey  for  those  who  did  not  participate. 

The  aim  of  this  thesis  is  to  act  as  a  foundation  for  ongoing  prediction  market 
research  within  the  Department  of  Defense  (DoD).  To  this  end,  we  answer  our  research 
questions. 

A.  RESEARCH  QUESTIONS 

1.  What  are  the  Key  Benefits  and  Potential  Limitations  of  Prediction 
Markets? 

Prediction  markets  have  a  proven  record  in  the  corporate  context.  The  key 
benefits  of  prediction  markets  include  their  dynamic  nature,  accuracy,  and  potential  for 
anonymous  revelations.  Prediction  markets  are  dynamic;  that  is,  they  are  capable  of 
efficiently  aggregating  information  in  a  continuous  or  ongoing  manner.  Additionally, 
prediction  markets  are  capable  of  aggregating  information  with  a  high  degree  of 
accuracy;  they  can  quickly  pull  together  information  spread  across  many  people  and 
places,  and  the  accuracy  of  the  results  can  rival  most  any  alternative  forecasting  method 
under  optimal  conditions.  Moreover,  prediction  markets  allow  users  to  express 
anonymous  personal  opinions  and  beliefs  on  specific  topics;  this  encourages  feedback  at 
all  organizational  levels  without  any  fear  of  management  or  peer  reprisal. 

The  chief  limitations  addressed  in  this  thesis  are  those  associated  with 
participation,  manipulation,  biases,  and  legal  restrictions.  Prediction  markets  require 
some  minimum  level  of  participation  to  avoid  stagnation.  Without  enough  active 
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participants,  some  traders  may  lose  interest  and/or  incentive  to  share  their  information 
with  others.  Furthermore,  manipulation  can  become  a  viable  concern  in  prediction 
markets  that  have  little  active  participation.  Additionally,  several  biases  can  affect 
trading,  and  thus  accuracy,  including:  over-optimism,  under-pricing  extremes,  long  shot, 
and  social  judgment.  Finally,  legal  restrictions  can  limit  some  organizations’  ability  to 
easily  or  efficiently  implement  and  manage  prediction  markets;  legal  restrictions 
generally  have  the  greatest  impact  on  real  money  prediction  markets,  as  government 
concerns  with  gambling  and  ethics  move  to  the  forefront. 

2.  What  Does  the  Current  Literature  and  Practical  Evidence  Suggest 
about  Prediction  Market  use  as  a  Forecasting  Tool? 

The  current  literature  highlights  practical  evidence  of  prediction  market  use  in 
various  industries.  Furthermore,  it  discusses  prediction  market  potential  and  the  arenas 
within  which  markets  should  be  considered.  Some  businesses  have  had  notable  success 
employing  prediction  markets.  These  companies  have  used  internal  prediction  markets  to 
help  forecast  key  events,  such  as  the  success  of  new  products,  future  profitability,  or 
mergers/acquisitions  within  their  industry. 

One  previously  noted  example  is  that  of  Hewlett-Packard’s  prediction  markets 
outperforming  traditional  printer  sales  forecasts  in  75%  of  the  observances  (Hanson, 
2003,  p.  107).  Additionally,  Google  has  enhanced  its  management’s  decision-making 
process  by  using  prediction  markets  to  forecast  how  many  consumers  will  use  their 
applications,  such  as  Gmail  (Dye,  2008,  p.  87).  Google  has  been  able  to  make  resource 
allocation  decisions,  such  as  server  capacity,  to  support  its  projected  number  of  users. 
Similarly,  pharmaceutical  companies  have  used  prediction  markets  to  determine  which 
drugs  in  their  experimental  process  have  the  best  chance  for  FDA  approval.  Thus,  they 
are  able  to  make  resource  allocation  decisions  based  on  those  drugs  that  have  the  highest 
probability  of  successfully  making  it  to  market.  Finally,  Abramowicz  (2003)  asserts  there 
are  times  to  believe  prediction  markets  may  provide  greater  efficiency  or  objectivity  than 
expert  forecasters:  "It  is  in  governmental  decisionmaking  [5/c],  however,  where  there  is 
the  greatest  reason  to  be  suspicious  of  experts,  either  because  of  external  influence  or 


because  of  ideological  agendas”  (pp.  20-21). 

86 


3. 


Can  Prediction  Markets  be  an  Effective  Tool  for  Navy  Force-shaping 
Forecasting? 


The  current  literature,  corporate  successes,  past  and  present  lessons  learned,  and 
the  growth  of  the  prediction  market  field  all  indicate  that  prediction  markets  could  be  an 
effective  tool  for  Navy  force-shaping  forecasting  and  decision-making.  However,  results 
of  this  project’s  experimental  prediction  market  are  inconclusive.  Several  constraints  and 
issues  caused  us  to  design  and  implement  the  experimental  market  in  a  less  than  ideal 
manner.  As  a  result,  we  can  speak  very  little  regarding  the  accuracy  or  efficiency  of  our 
markets.  Moreover,  issues  associated  with  participant  selection,  ongoing  participation, 
and  perceptions  of  ownership  reinforced  the  researchers’  preconceived  belief  that  certain 
cultural  and  organizational  hurdles  may  adversely  affect  the  development  and 
implementation  of  an  official  Navy  prediction  market. 

The  researchers  still  believe  that  prediction  markets  can  be  a  positive  and 
powerful  decision-making  resource,  and  they  can  have  a  place  in  Navy  forecasting — with 
a  positive  organizational  culture  that  embraces  change  and  opportunity.  However,  what 
we  have  seen  to  this  point,  as  indicated  by  post-market  survey  results,  is  that  several 
individuals  elected  not  to  participate  because  they  did  not  understand  the  markets’ 
purpose  and  intent,  lacked  time  or  incentives  to  participate,  were  uninterested,  or  did  not 
believe  the  markets  were  spearheaded  by  Navy  organizational  leadership.  To  be  sure, 
these  issues  are  not  a  negative  reflection  upon  prediction  markets  or  upon  any  members 
associated  with  the  design,  implementation,  or  conduct  of  the  markets.  Rather,  such 
issues  are  an  expected  byproduct  of  the  research  and  experimental  process. 

The  aforementioned  issues,  highlighted  by  the  post-market  survey,  speak  more 
directly  to  underlying  issues  of  organizational  culture  and  proper  training.  More  effective 
training  might  have  curbed  some  of  the  sentiments  expressed  in  the  post-market  survey. 
Additionally,  the  Navy,  as  well  as  most  governmental  organizations,  is  known  for  having 
a  strong  and  unique  culture  steeped  in  tradition  and  heritage.  There  are,  however,  some 
commonly  known  and  experienced  negative  effects  associated  with  such  a  culture,  and 
the  inability  to  easily  accept  and  embrace  change  is  one. 
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Should  Navy  leadership  choose  to  use  prediction  markets  in  a  formal  and  official 
manner,  the  top  levels  of  the  chain-of-command  must  accept,  embrace,  and  encourage  the 
markets.  Moreover,  leadership  must  emplace  a  full  training  initiative  to  ensure  all 
members  of  the  organization  understand  the  markets’  purpose,  as  well  as  their  role(s) 
within  the  market.  Finally,  Navy  leadership  must  fully  understand  the  intricacies  of 
prediction  markets  to  employ  them  properly.  Lessons  learned  from  this  thesis  highlight 
the  extreme  difficulty  to  understand  what  organizational  issues  or  concerns  are  best 
assessed  by  prediction  markets.  More  specifically,  an  organization  may  forecast  specific 
sensitive  metrics,  such  as  endstrength,  but  prediction  markets  can  be  complicated  when 
attempting  to  forecast  these  metrics  because  of  several  problems: 

•  Inconsistencies  in  organizational  or  cross-organizational  terminology, 

•  Misunderstandings  or  misperceptions  among  organizational  leadership  or 
members  on  what  truly  is  or  should  be  forecasted,  and 

•  Difficulties  in  pinpointing  the  means  by  which  organizations  officially 
measure  a  given  metric,  while  ensuring  markets  are  appropriately  closed 
and  contract  payouts  are  noncontroversial. 

4.  In  what  Areas  should  the  Navy  Consider  using  a  Prediction  Market? 

Since  the  results  of  the  experimental  prediction  market  are  inconclusive,  one 
cannot  say  with  certainty  how  the  Navy  should  use  prediction  markets.  Moreover,  this 
thesis  is  the  first  of  several  to  examine  the  potential  for  Navy  prediction  markets. 
Subsequent  theses  will  focus  more  on  practical  applications  and  developments  to  meet 
the  Navy’s  real-world  needs.  Thus,  the  results  of  those  theses  should  be  more  conclusive 
and  offer  greater  insight  in  this  respect. 

In  addition,  it  is  worth  reiterating  major  participatory  challenges  the  Navy  will 
face  in  using  prediction  markets,  as  discussed  in  Chapter  VIII.  First,  the  Navy’s 
prediction  market  claims  will  be  highly  specialized — such  as  claims  on  endstrength,  re¬ 
enlistment  bonuses,  recruiting  goals  and  other  issues  that  are  not  common  to  the 
layperson — and  they  will  require  a  degree  of  expertise  for  individuals  to  trade  with  any 
certainty  within  a  given  Navy  market.  As  a  result,  potential  participants  may  perceive  that 

they  are  at  a  disadvantage  in  a  market  because  other  traders  possess  either  privileged 
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information  or  the  necessary  degree  of  specialization,  while  they  themselves  do  not. 
Second,  traders  may  believe  that  their  participation  is  futile  or  irrelevant  since  market 
parameters  can  easily  change.  Decision  makers  can  affect  prediction  markets  by  changing 
policies  or  conditions  that  are  directly  associated  with  a  given  market;  when  this  occurs, 
trader  positions  are  undermined  within  the  affected  market.  Therefore,  the  Navy  should 
avoid  using  prediction  markets  for  predicting  outcomes  that  have  parameters  that  are 
likely  to  change. 

In  light  of  the  major  challenges  facing  prediction  market  utilization,  the  Navy  can 
utilize  prediction  markets  to  forecast  many  issues  of  concern.  The  key  to  successfully 
employing  the  markets  lies  in  designing  effective  and  measurable  market  contracts 
pertaining  to  the  issue(s)  of  concern,  developing  a  useful  participation  pool,  and  eliciting 
ongoing  participation.  Employed  properly,  prediction  markets  should  be  able  to  assist 
decision  makers  in  most  issues  of  concern. 

Post-market  survey  results  offer  some  specific  examples  of  key  forecasting  issues 
for  which  the  Navy  might  consider  utilizing  prediction  markets.  In  addition  to 
endstrength  and  annual  enlisted  accession  goals,  as  covered  by  this  thesis’s  experiment, 
survey  respondents  cited  the  following  specific  Navy  issues  for  future  prediction  markets: 
advancement  opportunity,  retention  rates,  and  attrition  rates.  The  following  are  some 
manpower  outcomes  we  believe  could  potentially  work  in  a  prediction  market: 

•  Advancement  opportunities; 

•  Attrition  rates — unlike  recruiting  and  retention  outcomes,  attrition  would 
not  likely  be  subject  to  policy  changes; 

•  Whether  the  Navy  will  change  certain  recruiting  and  retention  incentives, 
for  example: 

o  Increase  (or  decrease)  an  enlistment  bonus 

o  Increase  (or  decrease)  SRB’s  for  particular  ratings 

o  Increase  (or  decrease)  the  limit  on  enlistees  without  a  high  school 
diploma 


89 


5.  In  Conducting  a  Pilot  Prediction  Market,  What  Lessons  Learned  can 
we  Provide  to  the  Navy  Regarding  the  Design,  Implementation,  and 
Utilization  of  Prediction  Markets? 

Participation  is,  perhaps,  the  most  critical  and  challenging  aspect  of  prediction 
market  implementation  and  conduct.  Traders  need  to  be  induced  into  participating  in  a 
prediction  market,  especially  when  the  participant  pool  is  small.  With  a  large  potential 
participant  pool,  a  low  participation  rate  is  not  necessarily  bad.  An  efficient  market, 
rather,  depends  upon  a  critical  mass  of  active  traders.  If  an  organization  with  limited 
potential  participants  introduces  a  prediction  market,  a  higher  participation  rate  is  more 
critical  than  in  an  organization  with  a  larger  potential  participant  pool.  In  any  case, 
participant  selection  should  be  more  inclusive  than  exclusive.  It  is  important  to  include 
people  from  varying  organizational  levels  and  departments  so  the  prediction  market  may 
incorporate  the  information  afforded  them  by  their  unique  perspectives. 

Additionally,  there  is  no  such  thing  as  a  “one-size-fits-all”  design  for  all 
organizations.  As  described  in  Chapter  V,  there  are  many  considerations  when 
determining  what  type  of  market  is  most  useful,  given  organizational  preferences  and 
parameters.  The  primary  considerations  are: 

•  How  large  is  the  potential  participant  pool? 

•  What  type  of  information  is  the  organization  forecasting? 

If  the  potential  participant  pool  is  small,  an  automated  market  maker  is  necessary 
to  promote  liquidity  and  avoid  high  person-to-person  bid/ask  spreads.  Additionally,  the 
organization’s  selection  of  a  market  mechanism  should  depend  on  the  type  of  information 
the  organization  desires  to  forecast.  If  an  organization  is  forecasting  mutually  exclusive 
or  discrete  outcomes,  MSR  is  an  appropriate  mechanism.  However,  MSR  does  not  allow 
organizations  to  forecast  continuous  organizational  metrics;  the  CD  A  or  DPM 
mechanisms  are  more  appropriate  when  organizations  want  to  pinpoint  a  continuous 
number. 

Group  training  is  necessary,  as  it  can  preclude  or  mitigate  confusion  or  doubt 
among  potential  participants.  Specifically,  organizations  should  use  group  training  for  the 
following  reasons:  to  introduce  potential  participants  to  the  prediction  market  interface 
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and  demonstrate  its  site  navigation,  to  inform  participants  of  various  trading  strategies, 
such  as  short  selling,  and  to  display  executive  sponsorship.  If  the  training  includes 
account  setup  and  interface  introduction,  participants  will  have  fewer  reasons  for  not 
participating.  Furthermore,  group  training  may  provide  an  opportunity  to  preemptively 
boost  participation  by  challenging  people  to  compete  against  one  another. 

Prediction  market  administration  requires  a  great  deal  of  time  and  effort,  and  it  is 
complicated  by  subtleties  such  as  question  phrasing.  It  is  difficult  to  develop  useful  and 
measurable  organizational  questions.  Additionally,  the  prediction  market  experiment 
proved  it  challenging  to  phrase  questions  unambiguously  so  people  with  varied 
backgrounds  all  understand  the  true  intent  of  the  question  asked.  It  is  equally  challenging 
to  derive  comparative  forecasts  to  measure  the  relative  accuracy  of  the  prediction  market 
forecasts.  Each  of  these  challenges  translates  into  additional  administrator  time  and  effort 
to  manage  a  market.  Moreover,  marketplace  administration  should  not  be  tasked  to  one, 
or  even  two  people.  Ideally,  organizations  should  designate  a  small  group  of  people  to 
manage  and  administer  the  marketplace — the  size  of  the  group  will  depend  on  the 
questions  involved,  as  well  as  the  nature  of  information  sought. 
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APPENDIX  A:  ACTIVE-USER  SURVEY 


Question 

number 

Questions  to  Ask 

Suggested  Question 
Type 

Responses 

1 

Do  you  feel  the 
Prediction  Market 
questions  were 
clear? 

Rate  one  item  on  a 

scale 

Comments  optional 

Rating  scale: 

Very  clear 

Somewhat  clear 

Neutral 

Somewhat  unclear 

Very  unclear 

2 

Did  you  feel  you 
had  expertise  in 
the  Navy-specific 
market  questions? 

Yes/No/Uncertain 

3 

What  group(s)  of 
Navy  personnel 
should  be  included 

in  future  N1 
prediction  markets 
to  help  better 
aggregate  force- 
structure 

information? 

Multiple  select 
multiple  choice 

Select  all  that  apply: 

Comments  optional 

Recruiters 

Manpower  analysts 

Budget  analysts 

Other 

4 

What  factors 
influenced  you  to 
initially 
participate? 

Multiple-select 
multiple  choice 

Select  all  that  apply: 

Management  influence 

Peer  pressure 

Comments  optional 

Relevant/interesting  questions 

Intrigue  in  Prediction  Markets 

Confidence  in  my  information 

Ability  to  provide  my  opinion  in  a 
non-confrontational  forum 

Competition 

Other 
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What  factors 

influenced 

your 

continued 

participation 

past 

the  first 

few 

weeks? 

Multiple-select 
multiple  choice 


Comments  optional 


Select  all  that  apply: 


Weekly  email  update 


Management  influence 


Peer  pressure 


Relevant/interesting  questions 


Intrigue  in  Prediction  Markets 


Confidence  in  my  information 


Ability  to  provide  my  opinion  in  a 
non-confrontational  forum 


Competition 


Other 


If  you  made  early 
trades  hut  reduced 
your  trading 

activity,  what  if 
anything,  limited 
your  participation? 


Multiple -choice 
multiple  select 


Select  all  that  apply: 


Not  applicahle 


Felt  questions  achieved  market 
equilibrium 


Lack  of  time 


Comments  optional 


Lack  of  incentive 


Uninterested 


Questions  were 

relevant/interesting 


not 


Lack  of  confidence  in  anonymity  or 
user  rights  protection 


Did  not  fully  understand  the  concept 
or  intent 


Technical/IT  issues 


Other 


What  incentives 
would  cause  you 
to  participate  in  a 
future  DoD/Navy 
prediction  market? 


Multiple-select 
multiple  choice 


Select  all  that  apply: 


None 


Preferred  parking  spot 


Lunch  with  VIP 


Comments  optional 


Movie  tickets 


Top  trader  t-shirt 


Recognition  at  an  HR  conference 


Monetary  reward 


iPod 


Other 
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8 

What  other  Navy 
topics  would  you 
find  interesting  to 
try  to  predict  using 
a  prediction 
market? 

Free  text 

9 

Do  you  believe 
you  understand  the 
concept  of  short- 
selling  (selling 
shares  you  don't 
have,  believing 
that  a  category  is 
priced  too  high)? 

Yes/No/Uncertain 

10 

Based  upon  your 
experience  with 
your  prediction 
market,  would  you 
participate  in  a 
future  prediction 
market? 

Yes/No/Uncertain 

Why  or  why  not? 
Comment  required 

11 

What  was  your 
understanding  for 
the  motivation  of 
this  Prediction 
Market  pilot? 

Single-select 
multiple  choice 
Comments  optional 

Select  one: 

NPS  Thesis  Project 

N1  Initiative 

Neither 

Both 
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APPENDIX  B:  NON-USER  SURVEY 


Question 

number 

Questions  to 
Ask 

Question  Type 

Responses 

1 

What  affected 

Multple-choice  multiple 

Select  all  that  apply: 

your  decision 

select 

Lack  of  time 

to  not 

participate? 

Lack  of  incentive 

Uninterested 

Comments  optional 

Questions  were  not  relevant/interesting 

Lack  of  confidence  in  anonymity  or  user 
rights  protection 

Did  not  fully  understand  the  concept  & 
intent 

Technical/IT  issues 

Other 

2 

What 

Multple-choice  multiple 

Select  all  that  apply: 

incentives 

select 

None 

would  cause 
you  to 

participate  in 
a  future 

Preferred  parking  spot 

Lunch  with  VIP 

Movie  tickets 

DoD/Navy 

Comments  optional 

Top  trader  t-shirt 

prediction 

Recognition  at  an  HR  conference 

market? 

Monetary  reward 

iPod 

Other 

3 

What  was 

Single-choice  multiple  select 

Select  one: 

your 

understanding 
for  the 

motivation  of 

NPS  Thesis  Project 

N1  Initiative 

Comments  optional 

Neither 

this  Prediction 
Market  pilot? 

Both 
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